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PERSPECTIVE  DISPLAYS  AND  FRAME-OF-REFERENCE:  THEIR 
INTERDEPENDENCE  TO  REALIZE  PERFORMANCE  ADVANTAGES  OVER 
PLANAR  DISPLAYS  IN  A  TERMINAL  AREA  NAVIGATION  TASK. 

Tyler  T.  Prevett,  M.S. 

Department  of  Psychology 
University  of  Illinois  at  Urbana-Champaign,  1994 
Christopher  D.  Wickens,  Advisor 

Perspective  projection  and  Frame-of-Reference  were  examined  and  hypothesized 
to  be  dependent  on  each  other  in  order  to  realize  performance  advantages  over  planar 
displays.  Forty  pilots  flew  simulated  instrument  approaches  on  four  possible  paths  to 
the  same  runway.  Eight  subjects  were  assigned  to  each  of  the  following  display 
conditions:  perspective  with  egocentric  viewpoint  location,  perspective  with  a  close 
exocentric  viewpoint  location,  perspective  with  a  middle  distance  viewpoint  location, 
perspective  with  a  far  viewpoint  location,  or  planar  array.  Results  reveal  that 
egocentric  perspective  displays  support  better  tracking  performance  than  either  planar 
or  exocentric  perspective  displays,  while  the  middle  distance  exocentric  display 
supported  better  awareness  performance  than  the  planar  or  egocentric  displays.  This 
shows  that,  indeed,  the  advantage  of  perspective  displays  over  planar  displays  is 
dependent  on  the  type  of  performance  measure,  as  well  as  the  viewpoint  location. 
Further,  a  significant  advantage  in  awareness  performance  was  realized  with 
northbound  paths,  while  tracking  performance  was  not  effected  by  direction  of  travel. 
Results  are  discussed  in  terms  of  map  display  design  for  electronic  approach  plates. 
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1.  INTRODUCTION 
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The  most  difficult  and  perhaps  the  most  dangerous  aspect  of  flight  is  approaching  and 

•s. 

completing  a  safe  return  to  the  earth.  This  danger  ofapproach  and  landing  is  apparent  from 
the  accident  ratio;  while  compromising  only  3%  of  total  flight  time,  landing  and  approach 
account  for  47%  of  total  aircraft  accidents  (O'Hare  &  Roscoe,  1990).  A  review  of  data 
compiled  by  NASA  via  the  Aviation  Safety  Reporting  System  (ASRS)  accomplished  by 
Williams,  Tham  and  Wickens  (1992)  revealed  that  geographic  disorientation  can  be  a  major 
factor  in  such  mishaps.  As  a  case  in  point,  consider  the  crash  of  TWA  fit.  514  in  1975,  where 
the  aircraft  crashed  into  a  mountain  near  Dulles  airport  as  a  result  of  the  crew  misinterpreting 
the  vertical  terrain  features  with  respect  to  the  approach  path  and  ground  features.  Due  to  the 
potentially  lethal  prospects  of  terrain  proximal  flight,  particular  attention  has  been  given  to  the 
design  and  implementation  of  different  display  formats  in  order  to  give  pilots  a  better  method 
of  acquiring  awareness  of  surroundings  and  situations  important  to  their  flight.  Of  course, 
redesign  and  implementation  of  this  navigational  information  is  only  possible  by  first 
examining  some  of  the  problems  with  traditional  methods  of  presenting  such  information. 

Currently,  most  pilots  refer  to  paper  depictions  of  flight  information  produced  either  by 
Jeppesen  Sanderson,  Inc.,  or  the  United  States  Government.  Figure  1  is  an  example  of  such 
a  paper  depiction  of  flight  and  approach  mformation.  This  particular  instrument  approach 
plate  was  taken  from  a  Department  of  Defense  booklet  to  be  used  by  USAF  pilots.  The  paper 
formats  of  instrument  approach  plates  (LAP)  provided  in  booklets  by  either  the  National 
Oceanic  and  Atmospheric  Administration  (NOAA),  or  by  Jeppesen  Sanderson,  INC.,  are  used 
by  90%  of  US  transport  pilots  (Kuchar  &  Hansman,  1993a).  A  useful  overview  of  actual  plate 
usage  and  content  is  presented  by  Mykityshyn  &  Hansman  (1991),  and  an  excellent  overview 
of  cognitive  issues  associated  with  IAP  usage  is  presented  by  Clay  (1993).  These  paper 
approach  plates  have  been  criticized  for  various  reasons  such  as  their  clutter,  inability  to 
support  rapid  retrieval  of  information,  disruption  of  pilots  ability  to  read  the  information,  and 
inability  to  provide  awareness  of  terrain  features  (Mykityshyn  &  Hansman,  1991;  Rate  & 
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Wickens,  1993).  Rate  and  Wickens  (1993)  also  state  that  efforts  to  improve  use  of  these  paper 
plates  have  not  been  satisfactory,  necessitating  a  new  approach  of  plate  presentation.  The 
advantages  of  electronic  representation  of  this  information  are  becoming  increasingly  salient 
with  new  generations  of  graphic  software  and  hardware,  and  represent  an  attractive  alternative 
to  the  traditional  paper  renditions.  Increased  efficiency  of  updating  information,  easy  access  to 
many  different  displays,  and  real  time  placement  of  the  aircraft  within  the  environment  are 
some  of  the  primary  advantages  possible  with  an  electronic  system  of  navigational  information 
presentation  (Mykityshyn  &  Hansman,  1991).  In  fact,  some  aircraft  already  use  a  form  of 
electronic  visual  representation  for  navigation,  Boeing  757/767  and  747-400  aircraft  have  an 
electronic  horizontal  situation  indicator  (EHSI)  which  represents  a  birds-eye  two-dimensional 
depiction  of  navigation  information  and  is  the  primary  navigation  display  in  glass-cockpit 
aircraft  (Kuchar  and  Hansman,  1993a).  Thus,  the  applicability  of  geographical  display 
equipment  is  already  being  realized,  but  only  in  a  small  portion  of  the  aviation  community. 

For  these  reasons,  several  researchers  have  attempted  to  compare  various  aspects  of 
electronic  map  displays  in  order  to  achieve  an  electronic  approach  plate  which  best  supports 
flying  tasks.  However,  electronic  display  by  means  of  a  computer  screen  also  provides  many 
degrees  of  freedom  for  the  actual  display  space  such  as:  the  amount  of  terrain  to  include,  the 
frame  of  reference  to  adopt,  the  dimensionality  (2D/3D)  to  use,  or  even  the  number  of  colors 
to  represent.  Hence  decisions  for  graphical  design  must  be  made.  Given  a  certain  type  of 
display  presentation,  another  question  arises;  how  should  pilots  performance  be  assessed?  In 
the  next  few  sections  those  basic  issues  of  presentation  and  assessment  will  be  examined  in 
light  of  research  already  accomplished.  The  focus  of  the  present  study  will  then  turn  to 
examining  inconsistencies  between  the  results  of  previous  studies  and  manipulation  of  some  of 
the  specified  design  parameters  of  electronic  display  in  used  for  navigation  tasks  in  an  attempt 
to  reconcile  those  inconsistencies. 
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Assessment  Methods 

Since  examination  of  design  issues  will  be  based  on  certain  measures  of  pilot 
performance  a  brief  definition  and  description  of  thosfc  assessment  methods  prior  to  using  th-m 
as  a  basis  for  design  parameter  specification  is  appropriate.  This  section  describes  two 
commonly  used  methods  of  assessing  pilot  performance  and  the  importance  of  those  measures. 
Local  Guidance 

In  any  approach  to  landing,  there  is  an  imaginary  ideal  flight  path  which  pilots  try  to 
follow  to  the  runway.  Maintaining  an  aircraft  on  this  flight  path  is  termed  Local  Guidance  and 
is  considered  a  form  of  closed-loop  tracking.  LG  tasks  are  those  that  link  flight  performance 
and  location  to  a  desired  performance  or  loccation,  and  can  be  measured  by  deviations  from  the 
flight  path.  For  example,  if  an  aircraft  was  supposed  to  be  at  point  (x,y,z)  at  a  given  time  (t), 
local  guidance  assessment  is  easily  seen  to  be  deviation  from  point  (x,y,z)  at  the  given  time  (t). 
Alternative  ways  to  compute  this  deviation  are  realized  by  taking  either  the  Root  Mean  Square 
(RMS)  of  that  deviation,  or  the  mean  absolute  value  of  the  deviation.  Both  of  these  measures 
remove  the  positive  or  negative  sign  and  leave  only  an  absolute  measure. 

Another  form  of  local  guidance  is  navigation  between  two  points.  In  this  case,  a  start 
point  and  end  point  are  defined  such  that  the  pilot  must  navigate  from  one  to  the  other. 
Interpoint  navigation  such  as  this  is  very  similar  to  way-point  navigation,  where  pilots  actually 
fly  from  one  magnetically  defined  point  to  another.  Assessing  pilot  performance  using  this 
measure  consists  of  either  recording  time  between  points,  #  of  points  reached,  or  deviation 
from  a  path  between  the  points  by  methods  stated  above. 

While  defining  the  construct  of  LG  is  relatively  easy,  abstracting  the  relative 
importance  of  LG  error  information  on  different  axes  is  more  difficult.  How  important  is  20 
feet  of  lateral  deviation,  or  20  feet  of  vertical  deviation?  It  may  be  argued  that  vertical  errors 
are  naturally  more  important  because  there  is  simply  less  vertical  space  as  compared  to  lateral 
space.  Fadden,  Braune,  and  Wiedemann  (1991)  point  out  that  vertical  information  needs  to  be 
presented  on  a  larger  scaling  resolution  due  to  the  relatively  small  differences  in  vertical 
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changes  as  compared  to  relative  lateral  changes  in  the  same  time  frame,  and  that  vertical 
instruments  must  contain  some  level  of  prediction  to  be  useful  in  enhancing  pilot  awareness, 
or  even  for  maintaining  a  specific  altitude.  The  different  importance  levels  of  lateral  and 
vertical  information  are  also  apparent  in  the  consequences:  loss  of  vertical  positioning  could 
result  in  a  crash,  while  the  general  consequence  for  lateral  loss  of  positioning  is  simply  getting 
lost  within  the  airspace  (although  the  two  dimensions  will  interact  in  mountainous  terrain.) 
Additionally,  the  importance  of  such  deviations  is  dependent  on  exactly  where  the  aircraft  is  in 
its  flight.  If  the  desired  flight  path  is  10  feet  above  the  ground  at  a  given  time,  and  the  aircraft 
is  20  feet  below  the  flight  path,  then  the  vertical  deviation  is  critical,  as  it  places  the  aircraft  10 
ft  below  the  ground.  However,  the  same  vertical  deviation  at  an  elevation  of  5000  ft  is  not  as 
important.  A  lateral  deviation  of  100  feet  while  in  flight  is  simply  not  as  important  as  a  lateral 
deviation  of  100  ft  when  on  the  ground  due  to  the  close  proximity  of  other  aircraft,  buildings, 
and  runway  borders.  In  general,  the  closer  the  aircraft  is  to  the  landing,  the  more  important 
flight  path  deviations  become. 

Situation  Awareness 

With  advanced  technology  aircraat,  it  is  possible  for  pilots  to  take  off,  navigate  to  their 
destination,  and  land  without  eveer  knowing  where  they  are  in  relation  to  any  terrain  features  or 
other  aircraft.  This  does  not  appear  to  be  a  safe  or  wise  way  to  accomplish  a  flight  mission. 

For  example*  what  would  pilots  do  if  confronted  with  unexpected  severe  weather,  an  intruder 
into  their  flight  path,  or  some  major  mechanical  malfunction?  Supporting  awareness  of 
surroundings  is  essential  to  providing  the  pilot  with  the  information  necessary  to  deal  with 
such  unexpected  events.  This  construct  is  termed  Situation  awareness  (SA),  and  is  used  by 
many  researchers  to  evaluate  pilot  performance.  Andre  et  al.,  (1991)  propose  that  the  loss  of 
SA  can  be  disastrous,  and  that  higher  levels  of  SA  are  reflective  of  safer  pilots.  SA  is  a 
difficult  concept  to  grasp,  and  even  more  difficult  to  actually  measure.  Pilots,  for  example, 
can  point  out  instances  or  students  with  excellent  SA,  but  have  a  more  difficult  time  telling 
you  exactly  what  SA  is  or  bow  to  objectively  measure  it  (Metalis,  1993).  While  an  exact 
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definition  of  this  construct  is  not  yet  fully  accepted,  over  the  past  decade  many  researchers 
have  made  strides  to  define  what  SA  is,  and  how  to  measure  it. 

Fracker  (1988)  compiled  a  list  of  different  definitions  of  SA  and  how  SA  relates  to 
military  flight.  One  such  definition  by  Tolk  &  Keether,  (1982)  defined  SA  as:  "The  ability  to 
envision  the  current  and  future  disposition  of  both  Red  and  Blue  [hostile  and  friendly]  aircraft 
and  surface  threats. "  A  definition  of  SA  taken  from  Whitaker  &  Klein  (1988)  indicates  that  SA 
is  the  pilot's  knowledge  about  his  surroundings  in  light  of  his  mission's  goals.  A  more  recent 
definition  has  been  formulated  by  Crabtree  et  al.,  (1993)  who  suggest  that  SA:  "refers  to  the 
pilot's  ability  to  acquire  information  about  the  aircraft  and  the  flight  environment,  process  the 
information  and  respond  appropriately  in  real  time. "  it  is  important  to  note  here  that  SA  does 
not  always  require  the  application  of  knowledge,  making  this  definition  somewhat 
unacceptable. 

Endsley  (1993a)  clearly  captures  critical  features  of  the  above  definitions  with  the 
following  formal  definition  "the  perception  of  the  elements  in  the  environment  within  a  volume 
of  time  and  space,  the  comprehension  of  their  meaning  and  the  projection  of  their  status  in  the 
near  future". 

Contained  in  these  different  definitions  are  concepts  which  make  up  the  general  pattern 
of  how  to  locate  and  perhaps  how  to  measure  aspects  of  SA.  The  concepts  are  the  1. 
acquisition  2.  processing  and  3.  interpretation  of  relevant  flight  information.  However,  it  is 
important  to  note  that  SA  may  not  necessarily  involve  the  application  of  thatknowledge  as 
proposed  by  Crabtree  et  al.  (1993).  For  example,  if  an  approach  and  landing  are  completed 
without  incident,  then  much  of  the  information  which  the  pilot  could  have  used  in  case  of  an 
emergency  was,  in  fact,  not  used.  The  specific  definition  of  SA  then  depends  on  the  relevant 
flight  information,  and  not  necessarily  how  individuals  used  that  information.  If  "Red  and 
Blue"  or  "Enemy  and  Friendly"  aircraft  are  the  most  important  flight  information  to  the  pilot, 
the  most  immediately  life  threatening,  then  these  should  be  included  in  the  operational 
definition  of  SA.  Whereas  in  the  realm  of  commercial  terminal  navigation  areas,  the  more 
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important  flight  information  would  not  even  involve  "enemy  or  friendly  forces"  except  in 
terms  of  how  much  a  mountain  or  an  empty  ftiel  tank  could  be  an  "enemy"  or  a  "friend". 
Thus,  the  operational  definition  of  S  A  in  this  instance 'would  be  quite  different  than  the 
definition  of  SA  to  a  pilot  in  a  dogfight. 

In  contrast  to  the  numerous  definitions  and  applications  of  SA,  there  is  no  dispute  as  to 
universal  importance  of  SA  to  the  modem  day  flight  environment.  In  general,  all  aspects  of 
flight  are  dependent  on  the  pilot's  knowledge  of  his/her  aircraft,  and  the  surrounding 
conditions.  Understandably,  pilots  with  higher  levels  ofunderstanding  about  their  current 
situation,  and  future  implications  are  less  apt  to  undertake  a  dangerous  maneuver,  and  hence 
less  vulnerable  to  risk  of  incident.  When  pilots  find  themselves  in  a  novel  or  dangerous 
situation,  their  reactions  are  greatly  dependent  on  their  knowledge  of  the  state  of  the  world. 
Aside  from  flight  safety  considerations,  SA  is  considered  an  integral  part  of  mission 
effectiveness.  According  to  Endsley  (1993a)  SA  is  "a  key  ingredient  for  effective  decision 
making"  in  dynamic  environments  such  as  flight. 

Electronic  Projection 

One  key  to  the  support  of  both  LG  and  global  SA  is  in  the  choice  of  appropriate 
electronic  projection  techniques.  Electronic  display  technology  is  rapidly  becoming  more 
powerful  and  taking  up  less  space.  A  very  natural  extension  of  this  technology  lies  in  aircraft 
cockpits.  O'Hare  and  Roscoe  (1990)  propose  that  the  use  of  map-type  displays  for  navigation 
is  more  desirable  than  traditional  instrument  navigation  (IAPs)  because  the  displays  can  show 
the  real-time  direct  relation  between  the  aircraft  and  it's  surroundings.  However,  current 
electronic  navigational  information  systems  such  as  the  EHSI  in  Boeing  aircraft  usually 
incorporate  only  two  dimensions  of  information  on  the  two  dimensional  display,  usually 
neglecting  the  graphical,  spatial  depiction  of  height  which  is  displayed  using  only  digital 
numerical  readings,  or  traditional  analogue  altimeters  (McGreevy  &  Ellis,  1986).  Given  that 
display  technology  is  rapidly  evolving,  this  limitation  of  information  representation  need  not 
exist.  It  is  possible  to  create  an  image  or  projection  on  a  two-dimensional  screen  which 
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presents  integrated,  graphical,  three-dimensional  information.  Such  dynamic  graphical 
presentations  are  already  available  in  almost  all  home  video  games  and  home  computers. 

Projection  of  a  3D  scene  onto  a  2D  surface  is  usually  accomplished  using  any  of  a  host 
of  depth  cues  such  as  perspective,  texture,  relative  size,  height  in  the  visual  field, 
interposition,  binocular  disparity,  or  other  sources  of  depth  information  (See  Wickens,  Todd 
and  Seidler  (1989)  for  an  in  depth  explanation  of  these  and  many  other  depth  cues  applicable 
for  electronic  display).  One  of  the  more  complex  methods  of  achieving  a  realistic  3D 
interpretation  of  a  2D  screen  in  a  dynamic  setting  is  through  the  use  of  perspective.  In  fact, 
perspective  projection  of  a  3D  scene  onto  a  2D  electronic  display  represents  a  paradigm  used 
by  many  researchers.  However,  Kim  et  al.  (1987)  caution  that  such  a  form  of  presenting  three 
dimensions  of  information  requires  careful  design  of  the  projection  to  the  display  surface.  In 
particular,  the  method  of  projection  used  by  the  present  study  and  many  other  studies  is  called 
central  projection,  which  produces  images  similar  to  those  seen  by  our  eyes  or  through  any 
camera  view.  This  form  of  display  is  very  similar  to  simply  viewing  any  scene  on  a  television 
set  where  the  camera  filming  the  scene  is  at  the  actual  scene,  and  the  viewer  is  isolated  away 
from  the  scene.  More  formal  and  precise  explanations  of  perspective  geometry  are  presented 
by  Kim  et  al.  (1987),  and  Wickens,  Todd  &  Seidler  (1989)  as  well  as  many  other  researchers. 

In  the  present  study,  the  projection  of  the  3D  scene  is  dependent  only  on  the  geometric 
field  of  view  (GFOV)  and  the  viewpoint  (location)  of  the  camera  in  space.  In  this  context,  the 
GFOV  is  akin  to  the  zoom  function  of  the  camera  (wide  angle  to  telephoto),  and  the  viewpoint 
correlates  to  the  location  of  the  camera  with  respect  to  some  reference  point  such  a  runway  or 
an  aircraft.  As  seen  in  Figure  1.2,  an  x,y,z  coordinates  system  is  defined  at  the  reference 
point  and  the  location  of  the  camera  is  specified  in  terms  of  a  polar  coordinates  system,  where 
there  is  an  angle  away  from  the  polar  or  x  axis  (azimuth  angle),  an  angle  away  from  the 
horizontal  xy-plane  (elevation  angle)  and  a  distance  vector  from  the  reference  point  to  the 
camera. 
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In  different  studies,  either  local  guidance  performance,  SA  measures,  or  both  are 
examined  in  light  of  manipulations  to  these  design  parameters.  The  results  of  some  of  these 
studies  of  GFOV,  azimuth,  elevation,  and  distance  effects  are  reviewed  here  in  order  to 
provide  a  sense  of  understanding  of  the  implications  of  varying  levels  of  these  variables. 
GFQV 

The  Geometric  Field  of  View  (GFOV)  is  simply  the  visual  angle  of  the  image  as  seen 
from  the  viewpoint  or  camera.  To  put  it  another  way,  this  is  the  angle  of  the  visual  scene  at 
the  point  where  all  of  the  projectors  converge  on  the  camera  lens.  There  are  two  basic  effects 
due  to  changing  the  GFOV;  distortion  and  magnification/minifrcation  (Ellis  et  al.,  1985). 
These  effects  arise  whenever  the  cone  of  vision  adopted  by  the  viewpoint  is  not  the  same  the 
cone  of  vision  that  would  be  subtended  on  your  eye  if  positioned  at  the  same  location,  creating 
what  is  known  as  a  "virtual  space  effect"  (Wickens,  Todd,  &  Seidler,  1989).  Figure  1.3  shows 
a  graphical  interpretation  of  magnification/minification  effects.  Recall  the  analogy  of  the 
camera  sending  a  signal  to  the  television;  given  that  the  camera  is  positioned  at  the  same 
location,  when  the  GFOV  is  widened,  a  greater  area  is  depicted  on  the  television,  creating 
compression  of  the  objects  on  the  screen  and  hence  "minification".  Similarly,  when  the 
GFOV  is  narrowed,  there  is  expansion  of  objects  located  on  the  display  screen  and  a 
“magnification”  effect  is  present.  Roscoe  (1984)  discussed  the  perceptual  complexities 
associated  with  such  tunneling  of  the  visual  field,  and  indicated  that  one  of  the  effects  of 
magnification  is  the  removal  of  depth  cues  which  would  be  present  in  wider  fields  of  view, 
creating  the  perception  that  objects  are  further  away  than  they  actually  are. 

In  addition,  there  is  a  distorted  incongruence  of  motion  in  dynamic  displays  when 
either  of  these  effects  are  present,  this  is  due  to  the  viewer  assumption  that  the  display  view  is 
a  "window"  and  is  simply  being  looked  through,  as  opposed  to  the  reality  that  the  display 
actually  contains  more  or  less  of  the  scene  than  would  be  present  if  looking  through  such  a 
window  (Ellis  et  al.,  1985).  Much  empirical  evidence  regarding  the  implications  of  these  two 
effects  has  been  gathered  by  several  researchers. 
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In  a  study  conducted  by  Kim,  Ellis,  Tyler,  Hannaford  Sc  Stark  (1987)  subjects  were 
required  to  view  a  perspective  depiction  of  a  target,  and  track  that  target  with  a  dual  joystick 
apparatus.  Projection  parameters  were  manipulated  between  trials,  and  RMS  tracking  error 
was  recorded.  Tracking  RMS  while  viewing  from  five  GFOV  angles  (8, 12, 28,48, and  64 
degrees)  were  examined  and  revealed  that  tracking  error  generally  increased  with  increasing 
FOVs  (minification),  an  effect  which  can  be  attributed  to  decreasing  resolution  of  the  display 
space  in  terms  of  graphical  pixels  per  space  feet.  According  to  a  similar  study  in  1985  by  the 
same  group,  GFOV  only  effects  performance  substantially  when  greater  than  100  degrees 
(Ellis  et  al.  1985) 

McGreevy  Sc  Ellis  (1986)  used  a  static  representation  of  a  perspective  display  and 
required  subjects  to  judge  the  elevation  and  azimuth  angles  from  a  reference  cube  to  a  target 
cube  while  design  parameters  were  varied  between  trials.  GFOV  in  this  case  was  varied  from 
30-120  degrees  in  30  degree  increments.  Results  indicate  that  the  elevation  angle  from 
reference  to  target  was  consistently  overestimated,  this  effect  was  especially  pronounced  at 
narrow  FOVs.  In  addition,  a  GFOV  of  60  degrees  produced  the  least  overall  azimuth 
judgment  errors. 

Hendrix,  Bjomeseth,  &  Barfield  (1994)  examined  a  task  similar  to  the  reference-target 
cube  task  used  by  McGreevy  &  Ellis  (1986).  In  this  particular  experiment,  GFOV  had  little 
effect  on  azimuth  judgments,  and  smaller  GFOVs  resulted  in  small  elevation  judgment  errors. 

Barfield,  Rosenberg,  Han  &  Furness  (1992)  also  manipulated  GFOV  in  a  systematic 
way.  In  this  experiment  subjects  were  required  to  fly  an  aircraft  to  targets  at  different 
locations  and  altitudes  by  using  an  out-of  window  display.  RMS  error  was  recorded,  as  well 
as  flight  time  between  targets.  GFOV  was  kept  at  either  30,60,  or  90  degrees  for  different 
trials.  Consistent  with  the  tracking  of  Kim  et  al.,  Barfield  et  al.  found  that  the  smallest  (30 
degree)  field  of  view  supported  lowest  RMS  error  and  quickest  time  reach  the  next  target.  In 
this  study  tracking  performance  was  best  while  using  narrow  fields  of  view,  this  result 
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contrasts  McGreevy  &  Ellis’  (198$)  Endings  that  judgments  of  azimuth  and  elevation  were 
best  at  a  wider  60  degree  FOV. 

The  results  of  the  above  studies  can  be  summarized  and  interpreted  in  the  following 
way:  at  a  given  viewing  distance  from  the  reference  point,  smaller  GFOV  create  a  greater 
perceptual  resolution  of  the  relative  differences  between  objects.  This  effect  can  be  attributed 
to  the  fact  that  smaller  GFOV  results  in  magnification  of  the  scene,  and  greater  salience  of 
relative  differences  between  objects.  Also,  it  is  important  to  note  that  these  performance 
results  are  contingent  on  the  location  of  the  objects,  and  distortion  due  to  the  virtual  space 
effect  causes  relative  perceptual  misinterpretations.  Simply  put;  distortions  due  to  the  virtual 
space  effect  are  greater  when  objects  are  closer  in  the  actual  scene  than  the  reference  point  or 
further  from  the  center  of  the  graphical  display.  It  is  interesting  to  note  that  none  of  the  above 
experiments  reported  differences  in  actual  SA  as  a  result  of  varying  GFOV.  Performance  was 
assessed  in  terms  of  vertical  and  horizontal  judgments,  which  can  be  generalized  to  awareness 
of  relative  positioning.  One  would  assume  however,  that  smaller  GFOV  would  lead  to  worse 
SA  due  to  greater  magnification  and  a  reduced  overall  view  of  the  scene  in  question. 

Elevation  Angle 

This  parameter  is  the  angle  of  the  viewpoint  to  the  reference  point  relative  to  the 
horizontal  plane.  An  important  feature  of  the  elevation  angle  is  that  when  the  elevation  angle 
reached  90  degrees  (a  planar  or  birds-eye  view)  there  is  no  graphical  vertical  information 
represented  on  the  display  (complete  graphical  compression  of  the  vertical  axis).  In  this  type 
of  planar  display,  there  is  only  information  about  relative  horizontal  distance  from  the 
reference  point  to  any  objects  of  interest.  The  display  essentially  becomes  two-dimensional 
when  the  elevation  angle  is  90  degrees,  except  for  any  relative  size  depth  cues  which  might  be 
noticeable  to  the  viewer.  By  the  same  logic,  when  the  elevation  angle  becomes  0  degrees  with 
respect  to  the  reference  point  there  is  complete  depth  compression,  and  the  only  salient 
information  contained  in  the  display  is  in  regards  to  relative  vertical  distances.  In  this 
condition,  the  display  again  becomes  essentially  two-dimensional,  except  now  it  resembles  a 
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profile  display  instead  of  a  planar  display.  Most  of  the  research  reviewed  in  the  GFOV 
section  involved  manipulation  of  the  elevation  angle  as  well,  and  the  results  of  these 
manipulations  are  much  less  ambiguous. 

Ellis  et  al.  (1985)  used  elevation  angles  from  0  to  90  degrees,  in  15  degree  increments. 
The  best  three  dimensional  tracking  was  obtained  at  an  elevation  angle  of  45  degrees,  with 
performance  gradually  dropping  off  to  greater  values  at  0  and  90  degrees.  The  poor 
performance  at  0  and  90  degrees  was  attributed  to  the  dimensional  compression.  Consistent 
with  that  explanation,  at  both  extreme  elevation  angles  the  gain  in  performance  on  the  axis 
orthogonal  to  the  viewing  axis  was  not  offset  by  the  loss  in  performance  on  the  axis  parallel  to 
the  viewing  axis. 

The  results  found  by  Ellis  et  al.  (1985)  were  duplicated  almost  exactly  by  Kim  et  al. 
(1987),  who  used  the  same  tracking  task.  Incidentally,  little  variance  was  found  between 
elevation  angles  varying  from  30  to  60  degrees,  though  the  best  three-dimensional  tracking 
performance  was  found  at  45  degree  elevation  angles. 

In  the  tracking  task  studied  by  Barfield  et  al.  (1992)  only  used  two  levels  of  elevation 
angle;  30  and  60  degrees.  They  found  that  the  using  a  60  degree  elevation  angle  led  to  better 
performance  than  using  a  30  degree  elevation  angle.  This  result  was  evident  for  both  RMS 
error  and  in  the  time  to  fly  between  targets.  Also,  this  effect  was  mediated  by  GFOV,  such 
that,  the  advantage  for  the  60  degree  elevation  angle  in  both  tracking  and  time  between  targets 
was  maintained  at  smaller  GFOV,  but  disappeared  for  a  GFOV  of  90  degrees. 

Yeh  &  Silverstein  (1992)  conducted  an  extensive  study  which  involved  manipulations 
of  projection  parameters  as  well  as  different  depth  cues.  In  their  study,  subjects  were  required 
to  make  spatial  judgments  of  relative  depth  and  altitude  differences  between  a  reference  object 
and  a  target  object.  Viewing  orientations  [elevation  angles]  of  15,45,  and  90  degrees  were 
used  by  subjects  to  make  the  spatial  judgments.  Findings  included  an  interaction  of  viewing 
orientation  and  type  of  judgment.  Depth  judgments  were  slightly  faster  at  the  45  degree  than 
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the  15  degree,  but  altitude  judgments  were  much  slower  at  the  45  degree  viewing  orientation 
compared  to  the  15  degree  orientation. 

Hendrix  et  al.  (1994)  partially  duplicated  the  other  research  endeavors,  examining 
angle  judgments  between  a  target  and  reference  cube  at  depicted  elevation  angles  of  -15,  15, 
45,  75,  and  90  degrees.  They  found  that  both  elevation  and  azimuth  judgments  from  the 
reference  point  to  a  cube  of  interest  were  best  at  elevation  angles  between  15  and  45  degrees. 
Hendrix  and  his  colleagues  assert  that  the  advantage  for  these  lower  elevation  angles  is  most 
pronounced  at  small  GFOVs,  when  the  scene  is  magnified. 

Integration  of  the  above  studies  yields  a  straightforward  interpretation;  both  tracking 
and  position  judgments  using  perspective  displays  are  best  supported  by  elevation  angles 
around  45  degrees,  and  this  advantage  may  be  somewhat  dependent  on  maintaining  small 
GFOVs. 

One  important  aspect  of  these  studies  with  respect  to  the  tracking  measures  is  that  most 
used  an  absolute  deviation  from  the  flight  path,  and  did  not  separate  the  different  vertical, 
horizontal  or  longitudinal  errors.  Barfield  et  al.  (1992)  did  take  measures  of  both  horizontal 
and  vertical  errors,  but  only  reported  results  in  terms  of  an  average  RMS  error  from  the  flight 
path.  The  advantage  of  the  45  degree  elevation  angle  can  be  seen  as  a  curvilinear  compromise 
between  these  different  kinds  of  errors  due  to  the  graphical  compression  of  individual  axes  at 
different  elevation  angles.  In  the  one  extreme  where  the  elevation  angle  is  0  degrees,  lateral 
or  depth  tracking  errors  will  be  very  large,  and  will  get  markedly  better  with  small  increases 
in  the  elevation  angle.  By  the  same  token,  little  information  is  gained  in  the  depth  axis  by 
increasing  the  elevation  angle  if  it  is  already  greater  than  45  degrees.  At  the  other  end  of  the 
extreme,  when  the  elevation  angle  is  90  degrees,  there  is  compression  of  the  vertical  axis,  and 
vertical  tracking  errors  will  be  very  large.  In  direct  contrast  to  tracking  in  the  depth 
dimension,  when  the  elevation  angle  decreases  vertical  tracking  performance  becomes  much 
easier  due  to  a  dramatic  increase  in  the  amount  of  vertical  information  present  in  the 
perspective  display.  Overall  error  thus  reaches  its  minimum  at  around  45  degrees.  One 
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qualification  to  this  apparent  solution  concerns  the  relative  importance  of  accurate  tracking  in 
each  of  the  different  dimensions.  As  pointed  out  by  Fadden  et  al.(1991),  vertical  tracking  is 
very  often  more  important  than  lateral  tracking  and  this  compromise  of  45  degrees  might  not 
be  the  best  value  for  the  elevation  angle  if  the  relative  importance  of  the  different  dimensions 
is  taken  into  account.  In  the  instance  that  vertical  control  is  more  important,  the  ideal 
elevation  angle  would  be  slightly  lower  than  45  degrees  in  order  to  support  more  accurate 
vertical  control. 

Azimuth  Angle 

The  third  of  the  projection  parameters  which  define  the  perspective  display  is  the 
azimuth  angle,  or  the  angle  away  from  a  straight  ahead  view.  If  a  horizontal  plane  were 
defined  by  the  reference  point,  with  x  running  foreword-backward  and  y  running  side  to  side, 
the  azimuth  angle  would  be  the  angle  away  from  a  straight  x  alignment  required  to  intersect 
the  viewpoint  location  in  that  xy  plane.  Similar  to  the  elevation  angle,  when  the  azimuth  angle 
approaches  a  measure  which  is  orthogonal  to  one  of  the  x  or  y  axes,  there  is  compression  of 
that  axis.  At  the  extreme  case  the  axis  is  completely  compressed  and  relative  position 
information  from  the  reference  point  cannot  be  gained  in  the  compressed  axis  other  than 
through  the  use  of  relative  size.  It  is  important  to  note  that  this  azimuth  related  compression 
is  only  apparent  with  low  elevation  angles,  and  that  when  the  elevation  angle  is  great,  the 
compression  due  to  azimuth  becomes  minimal.  For  example,  at  elevation  of  0  degrees  above 
the  reference  point,  and  90  degrees  azimuth  from  the  polar  axis,  there  is  complete  compression 
of  lateral  (y)  axis,  but  at  an  elevation  angle  of  90  degrees  (birds-eye  view)  there  is  no 
compression  of  either  the  polar  or  lateral  axes. 

Ellis  et  al.  (1985)  found  that  for  the  three  dimensional  tracking  task,  the  best  tracking 
performance  was  obtained  at  0  degrees  of  azimuth,  where  the  viewpoint  was  located  directly 
behind  the  reference  point.  Azimuth  angle  was  varied  in  45  degree  increments,  and  little 
decrement  in  performance  was  observed  between  the  -45,0,45  degree  azimuth  angles.  Ellis 
and  his  colleges  also  note  that  tracking  performance  was  poorest  at  135  degrees.  Apparently, 
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there  were  two  strategies  of  accommodating  different  azimuth  angles,  mental  rotation,  and  a 
reversal  strategy,  and  at  an  azimuth  angle  around  135  degrees,  subjects  probably  experienced 
confhsion  regarding  which  strategy  to  use. 

In  their  intensive  study  of  perspective  design  parameters,  Kim  et  al.  (1987)  varied 
azimuth  in  addition  to  elevation  and  GFOV.  Tracking  error  due  to  different  azimuth  angles 
mirrored  the  effects  found  by  Ellis  et  al.  (1985),  specifically  that  RMS  error  was  the  smallest 
when  azimuth  angles  of  -45,0,  or  45  degrees  were  present  in  the  display. 

In  the  second  of  two  experiments,  Wickens,  Liang,  Prevett  &  Olmos  (1994)  conducted 
a  small  scale  study  in  which  azimuth  angle  was  fixed  at  either  0  or  30  degrees.  In  this 
particular  instance,  the  reference  point  was  an  aircraft  which  the  subjects  controlled  using  a 
joystick  with  two  axis  of  control.  Airspeed  was  held  constant,  and  the  subjects  were  to  guide 
the  aircraft  to  a  runway  landing  by  way  of  a  flight  path  displayed  in  the  visual  scene.  Results 
showed  little  performance  difference  between  the  zero  and  30  degree  azimuth  angles  in  terms 
of  either  horizontal  or  vertical  RMS  error  from  the  flight  path,  though  vertical  information  was 
augmented  by  the  incorporation  of  color  coding  to  depict  aircraft  height  relative  to  the  flight 
path.  In  the  absence  of  this  color  coding  it  is  likely  that  at  the  0  degree  azimuth  angle  vertical 
tracking  would  have  been  quite  poor  because  the  vertical  predictor  symbol  was  superimposed 
on  the  flight  path  and  would  have  been  difficult  to  perceive.  As  stated  by  Fadden  et  al. 

(1991),  the  presence  of  this  predictor  symbol  in  spatial  displays  is  essential  for  accurate 
vertical  tracking. 

It  would  appear  from  these  three  studies  that  increased  error  due  to  increased  azimuth 
is  negligible  when  the  azimuth  is  between  45  and  45  degrees.  Error  increases  at  a  rapid  rate 
when  these  boundaries  are  exceeded,  to  a  maximum  at  around  135  degrees,  and  then  declines 
slightly  as  azimuth  reaches  180  degrees,  at  this  angle,  the  viewpoint  is  positioned  such  that 
viewing  the  tracking  task  is  along  the  polar  axis,  but  the  lateral  control  input  is  exactly 
opposite  of  the  visual  feedback. 
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Distance  Vector 


The  last  projection  parameter  to  be  discussed  is  the  distance  vector,  defined  as  the 
length  of  the  vector  between  the  reference  point  and  the  viewpoint.  Varying  this  vector  in  a 
perspective  display  is  similar  to  simply  varying  the  scaling  of  the  map.  The  visual  effect  is 
much  like  the  magnification  or  nullification  of  the  scene  as  presented  by  the  GFOV  discussion 
above,  however  there  are  no  changes  which  create  variance  in  the  amount  of  visual  distortion. 
Very  often,  this  variable  is  manipulated  within  the  context  of  changes  to  the  GFOV  such  that 
the  amount  of  the  scene  present  in  the  display  remains  constant.  This  being  the  case, 
differences  in  tracking  error  or  in  SA  due  to  changing  the  distance  vector  of  the  viewpoint  are 
highly  dependent  on  the  GFOV. 

Ellis  et  al.  (1985)  varied  the  viewing  distance  from  2.5  to  160  centimeters,  and  the 
GFOV  was  adjusted  in  order  to  maintain  constant  scaling  such  that  close  distances  had  large 
GFOV,  while  far  distances  has  smaller  GFOV,  and  the  amount  of  the  scene  depicted  in  the 
display  remained  constant.  There  was  an  increase  in  tracking  error  when  the  distance  was 
extremely  small  (2.5  cm)  and  the  corresponding  GFOV  was  119  degrees.  The  other  distances 
did  not  show  any  difference  in  tracking  performance  as  a  function  of  the  position  of  the 
viewpoint.  It  is  important  to  note  that  the  distance  effects  were  mediated  by  varying  the 
GFOV  and  that  the  resulting  displays  were  graphically  similar  except  for  the  amount  of 
distortion  present. 

Previously  it  was  stated  that  Kim  et  al.  (1987)  found  increased  tracking  RMS  error 
with  increasing  GFOV.  This  result  was  duplicated  for  object  distance,  which  was  varied 
independently  of  GFOV,  such  that  increased  object  distance  led  to  increased  tracking  error. 
This  result  is  in  contrast  with  Ellis  et  al.  (1985)  because  the  distance  vector  in  the  former  study 
was  varied  independently  of  the  GFOV,  and  as  the  viewpoint  was  further  removed,  the 
resolution  of  differences  between  the  target  and  the  reference  probably  decreased  markedly. 

Yeh  &  Silverstein  (1992)  measured  spatial  judgments  for  different  perspective 
projection  distance  vectors  such  that  in  one  condition  subjects  had  a  close  viewpoint  and  in 
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another  condition  subjects  were  presented  with  a  more  distant  viewpoint.  Among  their 
interesting  results  was  an  apparent  interaction  between  the  relative  position  and  viewing 
orientation.  In  terms  of  just  the  distance  vector,  they  found  that  the  closer  viewpoint 
supported  better  depth  judgments  than  the  far  viewpoint,  and  conversely  that  the  further 
viewpoint  supported  better  altitude  judgments  than  the  ciose  viewpoint. 

There  is  relatively  little  research  in  the  area  of  viewing  distance,  and  there  is  an 
apparent  lack  of  research  as  to  the  effects  of  this  distortion  on  measures  of  SA.  It  appears, 
however,  that  increasing  distance  leads  to  increased  tracking  errors,  probably  due  to  lack  of 
resolution  with  respect  to  the  reference  point.  Also,  it  appears  as  though  varying  the  viewing 
distance  from  the  reference  point  results  in  curious  effects  on  judgment  in  different 
dimensions,  though  these  effects  have  only  been  observed  in  a  small  number  of  studies  (i.e. 
Yeh  &  Silverstein,  1992) 

Summary  of  Projection  Parameters  Effects 

An  overall  summary  of  the  effects  due  to  changing  projection  parameters  gives  a  more 
solid  interpretation  and  understanding  of  the  implications  of  perspective  projection.  It  seems 
that  close  distances  support  better  tracking  (LG),  presumably  due  to  increased  resolution.  This 
conclusion  is  reached  because  comparisons  which  held  resolution  constant  did  not  find 
advantages  for  these  close  distances.  In  fact,  Ellis  et  al.  (1985)  found  that  the  closest  distance 
led  to  poorer  tracking  due  to  the  increased  distortion.  These  same  close  distances  seem  to 
preclude  the  development  of  an  awareness  of  the  scene,  due  to  the  small  amount  of  the  visible 
scene  present.  In  this  instance,  if  the  GFOV  is  widened,  minimization  occurs  and  the 
resolution  with  respect  to  the  reference  point  is  again  reduced.  Also,  the  effects  of  elevation 
angle  and  azimuth  angle  both  seem  to  be  non-linear,  with  only  small  costs  to  tracking  up  to  45 
degrees  and  larger  costs  as  the  angles  increase.  It  is  reasonable  to  propose  that  these 
parameters  should  be  set  around  45  degrees  to  get  the  best  performance  compromise  in 
tracking  axes,  and  that  some  intermediate  level  of  distance  and  GFOV  should  be  set  to  offset 
the  effects  of  distortion  and  resolution  in  order  to  maximize  both  LG  and  SA  tasks. 
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Therefore,  in  the  present  study  elevation  and  azimuth  angles  are  set  at  45  degrees,  while 
distance  and  GFOV  are  manipulated. 

"w. 

Planar.  Profile,  and  Perspective  Displays 

As  previously  stated,  planar,  profile,  and  perspective  displays  are  qualitatively  different 
representations  of  terrain  that  can  be  induced  by  varying  projection  parameters.  The 
traditional  LAP  map  orientation  is  a  view  taken  from  an  elevation  angle  of  90  degrees.  Map 
depictions  of  this  nature  are  categorized  as  a  planar  display,  and  have  the  characteristic  that 
there  is  no  confusion  of  the  two  axes  presented,  though  there  is  complete  compression  of  the 
vertical  axes.  A  profile  display,  on  the  other  hand  is  taken  from  an  azimuth  of  90  or  -90 
degrees,  with  an  elevation  angle  of  0,  which  has  the  property  of  completely  and 
unambiguously  presenting  information  in  the  vertical  and  the  forward/backward  axes,  while 
creating  complete  compression  along  the  lateral  axes.  Perspective  displays  have  some  other 
combination  of  the  perspective  parameters  which  render  a  simulated  three-dimensional  surface 
with  some  degree  of  compression  and  confusion  of  the  different  axes.  These  tlirce  specific 
map  depictions  have  been  the  focus  of  comparison  for  several  studies  of  aviation  navigation. 
The  question  of  which  types  of  displays  support  better  pilot  performance  is  a  non-trivial  one 
which  bring  to  light  apparent  contradictions  as  to  which  types  of  displays  should  better  support 
either  tracking  or  awareness  tasks. 

One  principle  of  design  which  is  central  to  the  perspective  vs.  planar  or  profile 
question  is  the  Proximity  Compatibility  Principle  (PCP).  Wickens  &  Carswell  (In  press) 
provide  an  overview  of  the  psychological  foundation  of  this  principle,  and  its  relevance  to 
display  design.  In  the  aviation  domain,  the  task  of  flying  can  be  viewed  as  controlling  the 
aircraft  with  respect  to  altitude,  lateral  deviation,  and  speed.  These  three  tasks  are  highly 
related,  and  to  some  extent  they  depend  on  each  other.  For  this  reason,  the  PCP  stipulates  that 
the  integrated  tasks  of  flight  path  tracking  and  awareness  should  be  better  supported  by  an 
integrated  (perspective)  display,  while  focused  attention  tasks  such  as  identifying  specific 
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lateral  or  vertical  distances  or  relative  sizes  would  be  better  supported  by  more  separated 

displays  (planar  or  profile)  where  there  is  no  ambiguity  along  the  judgment  axis. 

*«• 

Wickens  &  Carswell’s  discussion  of  this  principle  describes  several  specific  advantages 
and  disadvantages  of  using  a  perspective  display  in  comparison  to  using  planar  type  displays. 
Reduced  scanning  is  one  benefit  of  integrated  perspective  display.  In  a  perspective  display 
there  is  only  one  image  of  the  aircraft  to  view,  while  in  an  array  of  2D  displays  there  are 
several  different  and  distinct  displays,  each  with  their  own  depiction  of  the  aircraft.  Thus,  an 
individual  trying  to  maintain  alignment  on  a  flight  path  while  using  an  array  of  planar  displays 
would  have  to  scan  between  the  several  displays,  while  an  individual  using  a  perspective 
display  would  not.  The  advantage  for  reduced  scanning  comes  in  the  form  of  time  savings 
from  the  elimination  of  scanning  across  several  displays. 

Another  advantage  of  perspective  displays  is  that  they  create  display  integration. 

Images  are  seen  in  a  more  natural  or  ecological  setting,  which  is  closer  to  what  an  individual 
would  expect  to  see  if  looking  on  the  scene  with  his  or  her  own  eyes.  This  principle  of 
pictorial  realism  was  proposed  by  Roscoe  as  early  as  1968.  Planar  display  arrays  require 
individuals  to  mentally  integrate  several  different  displays  in  order  to  gain  a  mental  image  of 
the  terrain  or  airspace,  while  this  integration  is  already  complete  in  perspective  displays.  The 
advantage  for  perspective  displays  results  from  the  decreased  effort  to  interpret  the  scene. 

However,  the  presentation  of  three  dimensions  of  information  on  two  dimensions  of 
display  results  in  the  sharing  of  display  dimensions,  which  in  him  creates  three  disadvantages 
of  a  perspective  display;  poor  line  of  sight  resolution,  line  of  sight  ambiguity,  and  axis 
confusion. 

Poor  line  of  sight  resolution  is  the  result  of  distant,  object  compression.  In  perspective 
displays,  objects  are  presented  as  more  distant  when  they  are  smaller.  Much  like  a  set  of 
railroad  tracks  converging  in  the  distance,  perspective  displays  create  a  convergence  of 
information  at  a  distance.  The  differences  in  depth  are  conveyed  by  fewer  pixels  on  the  screen 
than  the  differences  in  lateral  or  vertical  position.  Planar  displays  do  not  have  this  limitation, 
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as  all  objects  presented  in  a  display  maintain  their  actual  scaling,  no  convergence  is  present, 
and  resolution  of  objects  in  terms  of  display  space  is  not  dependent  on  that  objects  location  in 
the  actual  scene  being  displayed. 

The  disadvantage  of  line  of  sight  ambiguity  is  somewhat  related  to  poor  line  of  sight 
resolution.  This  disadvantage  results  from  uncertainty  of  position  along  the  line  of  sight 
adopted  by  the  perspective  display.  While  planar  displays  also  suffer  this  disadvantage  when 
there  is  complete  visual  compression  along  any  one  axis,  the  disadvantage  is  mediated  by  the 
fact  that  several  planar  displays  may  be  used  in  combination.  Planar  displays  overcome  this 
limitation  because  each  provides  unambiguous  presentation  of  2  axes  of  information,  therefore 
all  3  axes  of  information  can  be  discerned  without  compression,  though  in  different  displays. 

The  third  disadvantage  is  axis  confusion .  This  disadvantage  for  perspective  displays 
results  due  to  the  fact  that  each  of  the  three  axis  has  components  in  each  of  the  2  dimensions  of 
the  display  whenever  azimuth  angle  and  elevation  angle  are  not  0  or  90  degrees.  For  example, 
when  there  is  movement  in  the  vertical  dimension  of  the  display  screen,  that  movement  can  be 
interpreted  as  movement  in  any  of  the  three  axes  of  the  x,y,z  coordinates  system  and  vice- 
versa.  This  same  confusion  is  also  evident  with  horizontal  movements  in  the  display  screen. 

A  study  conducted  by  Haskell  &  Wickens  (1993)  demonstrated  some  empirical 
evidence  for  the  applicability  of  the  PCP  to  electronic  map  design,  and  documented  some  of 
the  advantages  and  disadvantages  of  perspective  displays.  Subjects  were  provided  either  a 
perspective  display  or  an  array  of  planar/profile  type  displays  in  order  to  pilot  an  aircraft 
through  a  desired  flight  path.  Their  findings  were  consistent  with  predictions  made  by  the 
PCP;  "for  a  three-dimensional,  spatial,  dynamic  task,  a  three-dimensional  perspective  display 
is  a  viable  and  possibly  preferable  design  alternative  to  an  array  of  planar  instruments." 

Haskell  &  Wickens  also  showed  that  ambiguity  along  the  line  of  sight  created  poorer 
performance  for  airspeed  control,  as  the  line  of  sight  for  the  perspective  display  was  directly 
along  the  flight  path,  which  was  the  axis  depicting  changes  in  airspeed. 
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In  contrast  to  the  previous  study,  Andre,  Wickens,  Moorman  &  Boschelli  (1991)  found 
that  the  disadvantage  of  line  of  sight  ambiguity  outweighed  the  advantages  of  perspective 
display.  In  their  experiment,  subjects  were  required  to- navigate  to  specific  locations  in  a 
three-dimensional  space  using  either  a  perspective,  or  several  2D  displays.  A  greater  number 
of  the  navigation  points  were  reached  using  the  2D  displays  because  subjects  had  difficulty 
discerning  when  they  had  reached  the  appropriate  destination  while  using  the  perspective 
display.  Situation  awareness  was  measured  in  terms  of  time  to  recover  from  disorienting 
events,  and  was  also  better  supported  by  the  planar  arrays  of  displays.  The  advantage  of 
integration  was,  however,  apparent  in  subjects  control  inputs,  as  simultaneous  vertical  and 
horizontal  inputs  were  made  more  often  while  using  the  perspective  display. 

A  study  conducted  by  Rate  &  Wickens  (1993)  contrasted  performance  while  using  a 
perspective  display  with  a  display  containing  a  profile  and  a  planar  display.  Measurements  of 
-performance  were  taken  in  terms  of  lateral  and  vertical  RMS  error  away  from  the  desired ■■ 
flight  path,  and  situation  awareness  was  assessed  in  terms  of  error  and  response  latency  to 
probed  questions  regarding  terrain  features.  Rate  &  Wickens  found  that  flight  path  guidance 
was  better  supported  by  the  array  of  2D  displays  rather  than  the  single,  integrated  perspective 
display.  This  apparent  contradiction  to  the  predictions  made  by  the  PCP  resulted  from  poor 
resolution  of  the  3D  display,  and  perhaps  some  axis  confusion. 

In  a  pair  of  experiments,  Wickens,  Liang,  Prevett  &  Olmos  (1994)  extended  the  work 
done  by  Rate  &  Wickens  (1993)  in  an  attempt  to  identify  modifications  which  would  bring 
perceptive  map  performance  up  to  the  level  of  2D  map  performance.  Despite  the  introduction 
of  several  display  improvements  and  additions,  Wickens  et  al.  found  that  planar/profile 
displays  maintained  a  substantial  tracking  advantage  in  the  vertical  axes,  while  performance 
along  the  lateral  axis  was  not  significantly  different  between  the  two  display  formats.  In 
addition,  situation  awareness  was  assessed  in  this  study  in  several  ways.  Frozen  screen  tasks 
were  implemented,  in  which  the  flight  simulation  was  halted  while  subjects  answered  several 
questions  concerning  their  awareness  of  the  surrounding  terrain..  Terrain  recall  was  also 
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assessed  with  a  post-flight  map  reconstruction  administered  to  each  subject.  The  investigators 
found  a  slight  advantage  for  perspective  maps  over  the  planar/profile  combination,  including 
quicker  response  times  but  generally  greater  error  rates.  These  results  indicate  an  absence  of 
exact  information  in  the  perspective  display,  but  faster  understanding  and  integration.  In  terms 
of  the  advantages  and  disadvantages  discusses  earlier;  results  from  this  study  failed  to  find  a 
tracking  advantage  for  the  perspective  display  despite  the  advantages  of  reduced  scanning  and 
increased  integration,  while  3D  integration  led  to  more  rapid  understanding  of  SA  tasks. 

Ellis,  McGreevy  &  Hitchcock  (1987)  examined  avoidance  maneuvers  executed  by 
pilots  flying  with  a  cockpit  display  of  traffic  information.  In  this  case,  subjects  were  required 
to  provide  an  appropriate  maneuver  to  avoid  an  air  traffic  incident  given  either  a  perspective 
display  or  a  planar  display.  Results  indicated  that  perspective  displays  supported  more  vertical 
avoidance  maneuvers,  fewer  unnecessary  maneuvers,  and  more  successful  avoidance 
maneuvers.  The  success  of  avoidance  maneuvers  can  be  attributed  to  the  fact  that  subjects  had 
a  more  realistic  and  natural  depiction  of  the  actual  situation,  which  required  no  mental 
integration  of  information.  In  addition,  Ellis  at  al.  found  that  the  advantage  for  perspective 
displays  was  not  evident  when  there  was  a  head  on  conflict,  which  can  be  attributed  to  the 
disadvantage  of  line  of  sight  resolution  or  ambiguity  along  the  line  of  sight.  The  results  of  this 
study  were  very  nearly  duplicated  by  similar  studies  accomplished  by  Bemis,  Leeds,  &  Winer 
(1988)  as  well  as  Wise,  Garland,  &  Guide  (1993).  All  of  these  studies  placed  the  planar 
displays  at  a  slight  disadvantage  because  they  were  not  implemented  with  an  analog 
representation  of  altitude,  making  altitude  change  rates  relatively  difficult  to  judge. 

Kuchar  &  Hansman  (1993b)  adopted  a  slightly  different  approach  to  comparing  profile, 
planar,  and  perspective  displays.  In  the  first  of  two  experiments,  terrain  avoidance  maneuvers 
were  examined  when  subjects  were  using  either  a  planar,  a  profile,  or  a  perspective  display. 

In  this  particular  example  subjects  were  given  an  electronic  horizontal  situation  indicator 
(EHSI)  and  an  attitude  indicator  as  well  as  one  of  the  three  display  types  in  order  to  pilot  their 
aircraft.  Four  terrain  hazard  levels  varying  from  none  to  severe  were  presented  to  subjects 
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and  subsequent  avoidance  maneuvers  were  recorded.  The  authors  found  less  controlled  flight 
into  terrain  when  subjects  used  the  profile  display,  while  subjects  were  equally  likely  to  impact 
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terrain  using  the  planar  or  the  perspective  display.  Kuchar  and  Hansman  advocate  the  use  of  a 
plan  and  profile  view  to  provide  unambiguous  lateral  and  vertical  information.  While  this 
finding  demonstrates  how  ambiguity  along  the  line  of  sight  can  lead  to  undesirable  outcomes, 
there  were  several  aspects  of  the  study  which  were  confounded.  Perhaps  the  most  significant 
of  these  was  the  use  of  only  a  0  degree  azimuth  view  for  the  perspective  display.  This  type  of 
view  completely  overlays  the  vertical  and  longitudinal  axes,  making  judgments  in  the  vertical 
axis  very  difficult.  The  effect  is  very  similar  to  the  effect  which  may  have  been  seen  when 
Wickens  et  al.  (1994)  used  a  0  degree  azimuth  angle,  except  that  Wickens  et  al.  had 
incorporated  color  coding  of  aircraft  height  information. 

The  overall  summary  of  the  performance  differences  between  the  2D  (planar/profile) 
and  3D  (perspective)  displays  remains  quite  ambiguous.  Neither  display  type  offers  a 
complete  advantage  in  both  tracking  and  situation  awareness.  In  addition,  results  from  these 
experiments  conflict;  some  found  tracking  or  awareness  advantages  for  perspective  displays, 
while  some  found  tracking  or  awareness  advantages  for  planar/perspective  displays.  The 
questions  arise:  why  the  conflicting  results?  When  do  advantages  associated  with  perspective 
displays  outweigh  the  disadvantages;  when  do  3D  (perspective)  maps  support  better 
performance  than  2D  (planar-type)  maps?  One  key  to  answering  this  question  appears  to 
emerge  from  examining  the  frame  of  reference  adopted  by  the  perspective  display,  and  this  is 
the  issue  we  consider  next. 

Frame  of  Reference 

In  addition  to  projection  design  considerations  based  on  the  geometry  of  the  projection, 
there  are  also  decisions  which  are  dependent  on  the  point  of  view  taken  by  the  display.  In  the 
aviation  navigation  domain,  this  Frame  of  Reference  (FOR)  consideration  varies  along  a 
continuum  from  a  view  where  all  the  presented  information  is  relative  to  the  fixed  terrain  (a 
world-referenced  frame  or  'WRF')  to  a  view  where  the  display  is  unique  to  the  individual 
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aircraft  position  and  orientation  (an  ego-referenced  frame  or  'ERF').  A  map  presented  in  a 
WRF  would  be  useful  to  all  aircraft  in  the  airspace,  while  a  map  presented  in  an  ERF  would 
only  be  useful  to  the  aircraft  it  is  taken  from.  Navigation  and  awareness  are  possible  when 
one  can  translate  ERF  information  into  WRF  information,  and  vice  versa.  This  translation  of 
information  has  been  addressed  at  length  by  both  Aretz  (1991)  and  Aretz  &  Wickens  (1992). 
The  actual  reference  frame  adopted  by  a  display  is  a  function  of  these  three  parameters:  the 
location  of  the  viewpoint,  the  relative  movement  of  the  viewpoint,  and  the  attitude  of  the 
viewpoint.  Figure  1.5  taken  from  Wickens  et  al.  (1994),  presents  these  different  levels  of 
egocentricity  which  define  the  FOR. 

Viewpoint  Location 

Where  the  viewpoint  is  located  in  the  three-dimensional  space  is  one  of  the  aspects  of 
frame-of-reference.  At  one  extreme  is  the  pilots  eye  view,  where  the  viewpoint  is  located  in 
the  aircraft,  and  depicts  exactly  what  a  pilot  looking  out  a  windscreen  would  see.  At  the  other 
extreme  is  a  viewpoint  location  very  far  from  the  aircraft,  looking  at  the  aircraft  from  a  gods- 
eye  view  in  the  context  of  its  surroundings.  The  pilots  eye  display  is  sometimes  referred  to  as 
an  inside-out  display,  or  an  egocentric  display,  while  the  gods-eye  view  is  sometimes  referred 
to  as  an  outside-in  or  exocentric  display.  This  frame  of  reference  difference  within  perspective 
displays  has  only  been  examined  by  one  notable  experiment. 

Barfield  et  al.  (1992)  examined  performance  differences  associated  with  either  an 
egocentric  or  an  exocentric  viewpoint  location.  They  found  that  the  pilots’  eye  display 
resulted  in  shorter  distances  traveled  between  targets,  and  that  the  advantage  for  this 
egocentric  display  was  roost  pronounced  at  larger  GFOV.  Also,  target  acquisition  time  was 
shorter  for  the  pilots’  eeye  display.  Barfield  and  his  colleagues  found  that  the  gods-eye 
(exocentric)  display  supported  more  accurate  reconstruction  of  the  map.  These  results  were 
attributed  to  display  compatibility  with  the  mental  models  pilots  had  of  either  the  LG  or  the  SA 
tasks.  It  is  important  to  note  here  that  the  pilots  eye  display  is  depicted  from  an  ERF  and 
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supported  better  tracking,  while  the  gods  eye  display  is  depicted  from  a  WRF  and  supported 
better  awareness  of  the  map  and  terrain. 

While  manipulation  of  viewpoint  location  between  conditions  has  not  been  the  primary 
focus  of  many  studies,  the  results  of  studies  which  have  adopted  different  viewpoint  locations 
can  be  compared  to  examine  the  extent  of  an  apparent  advantage  for  tracking  with  an 
egocentric  location  and  advantage  for  awareness  with  an  exocentric  location  as  demonstrated 
by  Barfield  et  al.  (1992).  Figure  1.4  provides  a  single,  multi-experimental  summary  of  the 
research  relevant  to  this  frame  of  reference  difference.  This  graph  compares  performance 
while  using  a  perspective  display  to  performance  while  using  some  form  of  planar  or  2D 
display,  and  reveals  the  presence  of  an  interaction  of  performance  measures  resulting  from  the 
assumption  of  different  frames  of  reference.  Performance  is  located  on  the  ordinate,  but  this 
measure  is  intentionally  left  ambiguous,  as  each  study  collected  different  measures  of 
performance.  The  abscissa  defines  the  appropriate  perspective  viewpoint  location  as  either, 
egocentric  (pilots  eye)  or  exocentric  (some  gods-eye  view),  this  dimension  is  also  somewhat 
ambiguous,  as  experiments  used  varying  features  or  dimensions  of  egocentricity/exocentricity 
to  define  their  frame  of  reference.  This  graphic  shows  that  Haskell  &  Wickens  (1993) 
compared  egocentric  perspective  view  to  planar  depictions,  finding  that  the  perspective 
supported  better  tracking  than  the  array  of  planar  instruments.  Rate  &  Wickens  (1993),  and 
Andre  et  al.  (1991)  compared  planar  vs.  perspective  exocentric  viewpoint  locations,  finding 
that  in  general,  planar  arrays  supported  better  tracking  performance  than  perspective  displays, 
while  awareness  measures  indicated  an  accuracy  cost  for  perspective  displays.  The  studies  by 
EUis  et  al.  (1987),  Bemis  et  al.  (1988)  and  Wise  et  al.  (1993)  propose  that  perspective  displays 
from  exocentric  viewpoints  support  better  avoidance  maneuvers  than  planar  displays,  while 
Kuchar  &  Hans  man  (1993b)  showed  that  this  advantage  is  dependent  on  the  type  of  incident  to 
be  avoided.  Synthesis  of  these  results  points  to  the  existence  of  a  tradeoff  between  local 
guidance  and  situational  awareness.  When  the  viewpoint  is  egocentrically  located,  guidance 
performance  is  increased  and  situation  awareness  decreased.  However,  when  the  viewpoint  is 
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exocentric,  guidance  performance  increases,  while  situation  awareness  may  or  may  not  be 
increased  depending  on  the  type  of  task  to  be  accomplished,  and  whether  that  task  requires 
knowledge  of  the  ERF  or  knowledge  of  the  WRF.  In'addition,  awareness  of  surroundings 
with  exocentric  viewpoint  locations  is  also  somewhat  dependent  on  viewpoint  motion,  which  is 
the  second  defining  factor  of  the  FOR. 

Viewpoint  Motion 

As  with  viewpoint  location,  viewpoint  motion  can  adopt  either  an  egocentric  orientation 
or  an  exocentric  orientation.  In  the  case  where  any  one  of  the  projection  parameters  (azimuth 
angle,  elevation  angle,  or  distance)  is  linked  directly  to  the  aircraft,  then  that  parameter 
becomes  egocentric,  and  when  that  parameter  is  linked  to  the  world  axes,  it  becomes 
exocentric.  Several  studies  have  examined  the  benefits  of  making  viewpoint  motion  fixed  with 
respect  to  either  the  aircraft  or  the  world,  and  the  results  are  very  similar. 

When  viewpoint  motion  is  fixed  with  respect  to  the  world,  the  actual  display  does  not 
change  with  aircraft  movements,  and  when  some  aspect  of  viewpoint  motion  is  fixed  to  the 
aircraft,  then  the  display  changes  whenever  the  aircraft  position  changes.  Comparing  the 
viewpoint  fixation  on  the  aircraft  to  the  viewpoint  location  fixation  on  world  axes  is  best 
exemplified  by  comparing  azimuth  angle  fixation  to  the  aircraft  or  the  world  in  an  electronic 
display.  Fixing  azimuth  angle  to  the  aircraft  creates  a  track-up  type  map  in  which  the  aircraft 
remains  stationary  and  the  world  rotates  around.  Conversely,  when  the  azimuth  angle  is  fixed 
to  the  world,  (usually  in  a  North-up  fashion)  the  world  remains  stationary  and  the  aircraft 
moves  around. 

Aretz  (1991)  examined  the  design  of  planar  electronic  maps.  In  this  experiment 
subjects  were  required  to  navigate  an  aircraft  through  terrain,  maintaining  awareness  of 
surroundings.  Aretz  proposed  that  a  north-up  depiction  aids  search  and  identification  of 
landmarks,  and  facilitates  communication  with  other  pilots  or  air  traffic  control.  Also,  the 
egocentric  azimuth  (track-up)  type  maps  aided  navigational  ability  and  hindered  formation  of 
terrain  awareness,  while  the  exocentrically  fixed  azimuth  (north-up)  map  benefited  the 
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formation  of  a  cognitive  map  of  the  terrain  but  hindered  accurate  ERF  judgments.  Rate  & 
Wickens  (1993)  also  demonstrated  a  tracking  advantage  for  rotating  maps  and  an  awareness 
advantage  for  fixed  maps.  Similarly,  Wickens  et  al.  (1994)  contrasted  egocentric  and 
exocentric  features  of  azimuth  with  2D  and  3D  maps,  finding  that  rotating  displays  supported 
better  flight  path  guidance  but  did  not  harm  awareness. 

Overall,  it  is  apparent  that  when  a  track-up  map  is  displayed,  because  azimuth  is 
egocentric  there  is  a  relative  tracking  advantage  over  a  north-up  or  exocentric  azimuth.  There 
may  also  be  an  awareness  advantage  associated  with  using  a  north-up  map,  but  this  result  is 
not  concretely  verified  in  all  instances.  The  relative  advantages  for  each  FOR  stems  from  the 
fact  that  navigation  requires  knowledge  of  both  the  aircraft  and  the  world.  In  track-up  maps, 
the  ERF  is  more  apparent,  therefore  ERF  tasks  such  as  tracking  are  well  supported.  With 
North-up  maps,  the  FOR  adopts  a  more  WRF  alignment,  better  supporting  tasks  which  depend 
on  acquisition  of  world  information  such  as  terrain  features. 

It  is  also  important  to  note  that  a  rotating  map  results  from  fixing  only  the  azimuth 
angle  to  the  aircraft,  and  that  other  aspects  of  motion  can  be  fixed  to  either  the  aircraft  or  the 
world.  For  example,  both  the  elevation  angle  and  the  distance  vector  can  also  be  fixed  to  the 
aircraft,  creating  a  somewhat  more  egocentric  display  with  respect  to  viewpoint  motion.  In  a 
perspective  display  where  all  three  of  the  location  defining  parameters  (elevation  angle, 
azimuth  angle,  and  distance  vector)  are  tied  to  the  aircraft,  a  “tether”  view  is  created  and  the 
camera  simply  follows  the  aircraft  along  its  flight  path  from  a  specific  distance  and  direction 
away  from  the  aircraft.  This  is  the  type  of  display  used  in  the  present  study,  and  viewpoint 
motion  is  considered  a  tether  which  is  egocentric  in  nature. 

Viewpoint  Attitude 

Another  aspect  of  egocentricity  is  viewpoint  attitude  which  can  be  fixed  to  aircraft  or 
remain  stationary.  When  attitude  is  fixed  to  the  aircraft,  it  is  considered  egocentric,  and  when 
attitude  is  exocentric,  the  camera  remains  stable  irrespective  of  aircraft  attitude  (see  Wickens, 
Haskell,  &  Harte,  1989).  While  these  relative  fixations  of  viewpoint  motion  are  not  a  focus  of 
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the  present  study,  it  is  important  to  understand  that  there  are  many  aspects  of 
egocentricity/exocentricity  which  have  not  been  systematically  examined.  In  the  present  study 
the  camera  attitude  was  exocentrically  fixed  such  that  the  aircraft  attitude  was  not  duplicated 
by  the  viewpoint. 

Hypotheses 

It  is  suggested  from  the  research  reviewed  above  that  the  tracking  advantage  of 
perspective  displays  over  planar  displays  is  mediated  by  the  FOR  adopted  for  the  perspective 
display,  though  this  effect  is  not  validated  in  a  single,  comprehensive  experiment.  When  the 
perspective  display  is  depicted  from  an  egocentric  viewpoint  location,  the  advantages  of 
integration  and  naturalness  facilitate  more  accurate  flight  path  tracking.  At  an  exocentric 
viewpoint  location,  disadvantages  of  ambiguity,  resolution,  and  axis  confusion  outweigh  the 
benefits  of  perspective  displays,  and  tracking  performance  is  worse  than  when  using  planar 
displays.  For  these  reasons,  we  hypothesize  that  when  comparing  perspective  to  planar 
displays,  tracking  will  be  better  with  an  egocentric  perspective  display,  but  worse  while  using 
an  exocentric  perspective  display. 

Performance  on  situation  awareness  tasks  appears  to  be  somewhat  more  ambiguous. 
There  is  an  advantage  for  perspective  displays  that  have  an  exocentric  viewpoint  location,  due 
to  a  more  natural  and  integrated  view  of  the  world.  However,  this  advantage  is  applicable 
only  to  certain  awareness  measures  that  depend  on  world-referenced  knowledge,  and  not  on 
ego-referenced  knowledge.  We  hypothesize  that  the  best  performance  on  world-referenced 
awareness  measures  will  come  from  a  perspective  exocentric  viewpoint  location  because  of 
logic  set  fourth  by  Wickens  (1990),  Woods  (1984)  and  summarized  by  Andre  et  al.  (1991): 
that  a  3D  view  of  the  airspace  provides  a  more  "natural”  or  "ecological"  representation  than 
conventional  plan  view  (2D)  displays  because  a  perspective  display  provides  a  more 
compatible  view  by  reducing  the  need  to  integrate  mentally  across  several  2D  displays. 

Given  the  two  hypotheses  above,  a  third  hypothesis  is  manifest  if  the  first  two  prove 
correct.  This  is  that  there  will  be  a  trade  off  of  guidance  and  awareness.  This  trade-off  will 
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come  in  the  form  of  increased  SA  and  decreased  tracking  performance  for  exocentric 
viewpoint  locations,  and  relatively  lower  SA  and  higher  tracking  performance  for  egocentric 
viewpoint  location. 

There  is  little  research  examining  guidance  and  awareness  performance  of  pilots  when 
the  display  used  is  an  exocentric  viewpoint  location  and  the  distance  vector  is  varied. 

However,  given  the  tradeoff  identified  above  between  GFOV  and  viewing  distance,  we  believe 
that  these  parameters  could  be  very  important  in  ptimizing  the  design  of  exocentric  3D 
displays.  Therefore,  a  further  goal  of  the  present  study  is  to  examine  the  effects  of  varying 
this  projection  parameter  when  the  display  viewpoint  is  exocentric. 

Design  and  Evaluation  Considerations 

ErojgcijQnDssign 

All  of  the  information  about  planar  vs.  perspective  and  egocentric  vs.  exocentric 
displays  leads  directly  to  the  question  of  design.  This  section  describes  some  of  the  major 
design  decisions  made  in  this  study  for  projection  design  parameters  and  for  aircraft  control. 

All  exocentric  locations  of  display  view  have  not  been  systematically  examined  to 
determine  the  maximum  amount  of  performance  which  can  be  derived  from  different  angles, 
or  even  from  different  distances.  For  this  reason,  we  made  an  informed  compromise  to  locate 
the  "camera"  for  our  3D  display  at  an  azimuth  angle  of  -45  or  45  degrees,  and  an  elevation 
angle  of  45  degrees,  because  these  angles  were  cited  as  having  the  greatest  advantage  for 
perspective  displays.  By  positioning  the  viewpoint  at  this  location  we  avoid  significant 
graphical  compression  of  any  of  the  three  axes  depicted  on  the  display  screen.  Also, 
viewpoint  motion  was  slaved  to  aircraft  motion  in  order  to  avoid  problems  of  resolution,  and 
to  keep  these  motion  variables  of  egocentricity  constant  across  all  display  conditions. 

Given  a  camera  location  at  fixed  angles  from  the  aircraft,  camera  movement  slaved  to 
aircraft  movement,  and  a  stable  camera  attitude,  we  afford  ourselves  one  degree  of  freedom  in 
exocentric  viewpoint  locations;  the  distance  from  the  aircraft.  By  examining  several  different 
distance  vectors  it  is  possible  to  get  a  clearer  picture  of  the  effects  of  distance  on  both  tracking 
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and  awareness.  Also,  the  form  of  the  hypothesized  trade-off  between  Situational  Awareness 
and  Local  Navigation  can  be  examined  in  a  systematic  way  by  looking  at  various  levels  of 
exocentric  viewpoint  locations . 

Deciding  which  levels  of  the  factor  to  test  proves  to  be  a  bit  more  complicated  than 
simply  dividing  the  maximum  distance  into  equal  portions.  For  example,  if  the  maximum 
display  distance  is  1000  ft  away,  testing  at  0  ft,  250  ft,  500  ft,  750  ft,  and  1000  ft  would  not 
be  completely  useful,  as  perception  of  displacement  and  motion  is  not  a  linear  function  of 
distance.  If  perception  can  be  related  to  the  visual  angle  subtended  to  the  eye,  than  the 
relationship  resembles  one  provided  by  Sanders  &  McCormick,  (1987)  to  define  visual  acuity. 

VA  (minutes)  =  3438*H 
D 

Or,  in  trigonometric  terms,  the  equation  would  be: 

VA  (degrees)  =  tan"*  (D/H) 

Where  H  is  the  height  of  the  object  and  D  is  the  distance  from  the  object.  In  our  experimental 
situation,  a  linear  change  in  distance  would  not  result  in  a  linear  change  in  visual  acuity.  This 
equation  gives  us  a  relation  of  the  VA  to  distance  which  is  clearly  non-linear,  but  rather  of  a 
logarithmic  form.  For  this  reason,  after  assuming  a  maximum  viewing  distance  of  70,000 
display  feet,  the  viewpoint  distance  was  lorithmicaly  set  at  10,000  ft,  25,000  ft,  and  70,000  ft. 
for  each  of  three  exocentric  viewpoint  locations  respectively. 

Another  display  design  item  which  is  of  importance  is  control  dynamics.  In  an  actual 
aircraft,  entering  a  roil  induces  pitch  down  of  aircraft  attitude.  For  this  reason,  backstick 
pressure  must  be  added  in  conjunction  with  roll  in  order  to  maintain  level  flight.  While  some 
studies  such  as  Andre  et  al.  (1991)  incorporated  this  flight  dynamic  into  aircraft  control,  while 
other  studies  such  as  Rate  &  Wickens  (1993)  did  not.  While  this  would  seem  a  minor  point, 
by  creating  two  separate  dimensions  of  control,  the  task  of  flight  guidance  essentially  became 
two  independent  tasks,  lateral  tracking  and  vertical  tracking.  The  Proximity  Compatibility 
Principle  clearly  indicates  that  advantages  are  to  be  gained  from  integrated  display  only  when 
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the  task  requires  integrated  control  (Wickens  &  Carswell,  in  press).  In  the  present  study  we 
have  cross  coupled  the  controls  in  an  effort  to  create  an  integrated  dimensional  control  task 
instead  of  two  separate  control  tasks. 

Another  area  of  concern  is  the  level  of  expertise  of  participants.  Andre  et  al.  (1991) 
used  flight  naive  subjects,  while  Rate  &  Wickens  (1993),  and  Wickens  et  al.,  (1994)  used 
pilots  with  little  or  no  instrument  flight  time,  and  Mykityshyn  &  Hansman  (1991)  used 
qualified  line  pilots.  The  problem  lies  in  the  control  strategies  adopted  by  pilots  at  different 
levels  of  expertise.  It  is  possible  that  instrument  pilots  have  different  control  and 
observational  strategies  which  impact  flight  control.  Possible  strategies  vary  from  merely 
observing  and  inputting  little  control  to  active  and  vigorous  control  and  scanning.  Research  in 
this  area  is  ongoing,  and  results  are  not  yet  formalized.  Since  this  study  involves  using 
electronic  IAP  charts,  the  decision  was  made  to  use  only  participants  who  had  some  instrument 
flight  training  in  order  to  avoid  making  judgments  based  on  possibly  different  control 
strategies  of  more  naive  subjects. 

Local  Guidance  Assessment  Methods 

In  the  present  study  deviations  from  the  flight  path  were  considered  more  desirable  than 
point  to  point  navigation  methods  due  to  the  nature  of  the  terminal  area  navigation  task. 

Taking  the  root  mean  square  (RMS)  error  at  specified  time  intervals  is  one  computational 
approach  to  measuring  such  deviations,  and  has  been  used  in  several  paradigms  of  research. 
The  actual  form  of  computation  involves  taking  the  square  root  of  the  average  squared 
deviation  from  the  flight  path.  While  this  measure  offers  an  overall  average  of  how  far  the 
aircraft  was  from  the  flight  path  without  regard  to  positive  or  negative  deviations,  certain 
limitations  are  associated  with  the  method.  The  main  problem  with  the  RMS  measure  is  that 
greater  deviations  can  have  disproportionately  more  weight  than  closer  deviations  if 
computational  methods  are  accomplished  in  different  orders.  To  rectify  this  problem,  one  can 
either  remove  the  overweighed  observations  as  outliers,  or  normalize  tbe  data  by  transforming 
it.  An  alternative  measure  of  flight  path  deviation  is  mean  absolute  error  (MAE),  which  is 
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simply  the  average  absolute  deviation  from  the  flight  path.  MAE  reduces  the  impact  of  large 
flight  path  deviations  without  totally  removing  their  presence.  In  the  present  study  we 
recorded  both  to  examine  post-hoc  differences  between  the  measures. 

SA  Assessment  Methods 

In  the  present  study,  the  domain  of  interest  is  the  terminal  area  around  the  runway,  and 
the  associated  approach  to  that  runway.  In  the  situation  where  a  pilot  is  approaching  a  landing 
there  are  several  items  which  are  of  importance.  One  important  item  is  the  flight  path. 
Therefore,  knowledge  of  ^tosition  away  from  the  flight  path  is  crucial  so  that  the  pilot  can 
control  the  aircraft  and  maneuver  it  back  to  the  flight  path  if  it  has  strayed.  Knowledge  of 
landmarks  is  another  important  issue  in  terminal  area  navigation.  Such  landmarks  include  the 
runway  as  well  as  major  geographic  features  of  the  surrounding  terrain.  Also,  clearances  to 
runway  landings  are  given  in  terms  of  "runway  numbers"  which  represent  magnetic  bearings, 
and  weather  information  is  given  in  world-referenced  terms,  so  that  some  knowledge  of  the 
world  canonical  directions  is  also  essential.  Fracker  (1988)  also  notes  that  pilots  stay  abreast 
of  their  immediate  situation  by  operations  carried  out  in  working  memory,  which  may 
incorporate  information  from  both  displays  and  from  a  long  term  retention  of  information. 
Since  the  current  study  will  be  looking  only  at  terminal  area  navigation,  the  operational 
definition  of  SA  that  we  use  here  is:  the  availability  of  flight  path  information,  canonical 
direction  information,  and  geographic  knowledge  from  both  short-term  and  long  term 
memory. 

Even  knowing  what  SA  is  does  not  guarantee  usefulness,  as  it  needs  to  be  a  measurable 
quantity  in  order  to  compare  different  levels  of  achievement.  This  proves  to  be  a  more 
difficult  undertaking  than  anticipated.  Several  models  of  assessment  and  improvement  have 
been  proposed  (  Ends  ley,  1993a;  Andre  et  al,  1993;  Fracker,  1988;  Metalis,  1993)  which 
include  knowledge  elicitation,  verbal  prottocol,  in-flight  report,  and  post-flight  questionnaires. 

Knowledge  elicitation  was  used  by  both  Wickens  et  al.  (1994)  and  Endsley  (1993a)  as 
methods  of  arriving  at  scores  for  measures  of  SA.  In  both  cases,  a  flying  task  was  temporarily 
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halted  in  order  to  ask  questions  regarding  the  participants'  knowledge  of  where  they  were, 
their  flight  stage,  and  other  systems.  This  method  will  also  be  used  in  the  current  study,  and 
since  it  can  be  accomplished  directly  by  the  computer^  it  eliminates  any  experimenter  bias.  In 
addition,  all  subjects  will  be  asked  the  same  questions  at  precisely  the  same  time  in  the  flight 
so  that  working  knowledge  of  the  terrain  and  flight  will  be  evaluated  for  each  participant  in 
roughly  the  same  situation. 

Post-flight  questions  on  terrain,  or  mission  requirements  is  an  approach  to  be  used  in 
the  present  study  as  well.  Whitaker  &  Klein,  (1988);  Wickens  et  al.,  (1994),  and  Arete  (1991) 
advocate  the  use  of  post  mission  queries.  In  the  latter  two  studies,  participants  were  asked  to 
reconstruct  a  map  of  the  terrain.  The  reconstruction  was  then  evaluated  by  several  individuals 
to  obtain  objective  metrics  of  correct  positioning,  structure,  and  numbers  of  objects  recalled. 
Fracker  (1988)  indicates  that  this  method  relies  on  situational  assessments  being  coded  into 
long-term  memory.  A  similar  map  reconstruction  will  take  place  in  the  present  study. 
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Figure  l.l:  Paper  IAP  taken  from  DOD  publication  for  USAF  pilots,  depicting  IAP  similar  to  those  presented  by 
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Distance  vector 


Pi<pire  1.2:  Depiction  of  parameters  essential  to  define  a  perspective  projection 
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Smal  QFOV 


Figure  1.3:  Magnification/Mlaificatioa  of  the  visual  scene  due  different  GFOV. 
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Figaro.  tA  Graphical  summary  of  pervious  research.  All  research  is  in  relation  to  2D  depictions  of  the  same 
information.  Dimensions  of  the  graph  are  left  general  in  order  to  accommodate  different  measurement  methods  of 
each  paradigm  of  research.  Accordingly,  the  performance  axis  it  made  up  of  tracidug  error,  time  to  fly  between 
targets,  and  various  measures  of  awareness  and  judgment  The  graph  points  to  the  existence  of  a  trade  off  between 
LG  and  SA  as  well  as  different  performance  levels  using  perspective  display  depending  on  the  Frame  of  Reference 
adopted. 
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Figure  2:  Egocentric  features  of  a  3D  display.  The  display  viewpoint  can 
be  fixed  in  earth  coordinates  (1),  or  slaved  to  move  with  the  aircraft  as 
if  on  a  '’tether"  (2).  Once  moving,  the  viewpoint  can  be  positioned  at  the 
pilot's  viewpoint  (like  a  HUD). (3),  or  any  distance  behind  (along  vector  "A"). 
If  behind,  it  can  be  positioned  at  any  altitude  above  the  aircraft  (along 
vector  B).  Furthermore,  the  viewpoint  can  have  either  a  fixed  elevation  and 
azimuth  angle,  or  angles  that  are  slaved  to  the  aircraft's  pitch  and  roll. 

The  variation  in  elevation  angle  (alpha)  is  shown  at  position  (2).  Not  ■ 
depicted  in  the  figure  are  possible  ranges  of  azimuth  angle. 


Figure  1 .5:  Depiction  of  variables  which  define  the  FOR ,  taken  firm  Wlckens  et  al.  (1994). 


2.  METHODS 


Variables 

The  design  consisted  of  a  4x4x5  blocked  design.  Independent  variables  include:  1) 
path  type  (East  or  West  side  of  the  runway,  flying  either  a  North  approach  or  a  South 
approach)  2)  Trials  (subjects  flew  four  trials  each)  3)  Display  type:  planar  array,  perspective 
with  egocentric  viewpoint  location,  or  one  of  3  perspective  displays  with  exocentric  viewpoint 
locations.  Path  and  trial  varied  within  subjects,  while  subjects  were  blocked  into  one  of  the 
five  display  conditions. 

Four  dependent  variables  were  measured:  1)  Lateral  RMS/MAE  from  the  flight  path 
2)  Vertical  RMS/MAE  from  the  flight  path  3)  Short  term  memory  of  situation  awareness,  4) 
Long  term  memory  of  situation  awareness.  RMS/MAE,  and  short  term  SA  were  evaluated  on 
each  of  the  four  approaches  subjects  flew,  while  long  term  SA  was  assessed  after  all 
approaches  had  been  completed.  Both  accuracy  and  response  time  were  measured  for  short 
term  SA  questions. 

Subifiste 

Forty  University  of  Illinois  aviation  personnel  participated  in  the  experiment,  each" 
receiving  payment  of  $5.00  per  hour  and  were  tested  for  approximately  1.5  hrs.  All  subjects 
were  pilots  with  private  licenses  and  some  instrument  time,  having  between  75  and  400  hours 
of  flight  experience  and  ranging  from  19  to  26  years  of  age.  Eight  subjects  were  randomly 
assigned  to  each  of  the  five  map  conditions,  each  flew  two  practice  approaches  and  all  four 
possible  approaches  to  the  runway  in  counter-balanced  order.  Subjects  received  the  same 
instructions  (contained  in  Appendix  A),  and  the  same  STM  and  LTM  awareness  tasks. 

Apparatus 

The  present  study  was  conducted  on  a  Silicon  Graphics  IRIS  workstation  with  a  16  inch 
diagonal  screen.  A  two-degree-of-freedom  joystick  was  attached  to  the  right  arm  of  the 
subjects’  chair.  This  joystick  controlled  the  lateral  and  vertical  axes  of  the  aircraft,  with 
standard  aviation  control.  Pitch  was  increased  with  back-stick  pressure,  and  roll  was  induced 
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with  side-stick  pressure.  Pitch  controlled  the  rate  of  change  of  altitude,  while  lateral  stick 
displacement  controlled  the  rate  of  roll.  The  aircraft  was  not  allowed  to  roll  past  90  degrees  in 
either  direction.  Pitch  was  cross-coupled  with  roll  such  that  inducing  a  roll  would  result  in 
proportional  and  simultaneous  pitch  down  of  the  aircraft.  Aircraft  speed  was  held  constant  at 
85  mph,  and  there  were  no  rudder  controls. 

Display 

Approach  path:  Figure  2. 1  depicts  a  planar  view  of  the  simulated  world  used  for  the 
study  in  which  there  were  four  possible  flight  paths  to  a  single  runway.  On  a  given  trial, 
subjects  flew  either  North  or  South  on  the  East  or  West  side  of  the  map.  Only  the  East  or 
West  side  was  depicted  on  the  display  at  a  time,  and  only  the  flight  path  being  flown  was 
presented  to  the  subject. 

Display  Conditions:  Eight  subjects  were  assigned  randomly  to  one  of  5  different 
display  conditions.  In  each  of  the  displays,  the  world  remained  the  same,  while  the  variation 
effected  only  the  view  of  the  world  seen  by  subjects.  Viewing  distance  from  the  aircraft  and 
FOV  were  systematically  varied  and  coupled  such  that  the  amount  of  the  world  in  view  at  any 
one  given  time  remained  the  same  between  the  display  conditions.  Within  each  display,  the 
viewpoint  remained  a  fixed  distance  from  the  aircraft,  viewpoint  motion  was  directly  linked  to 
aircraft  motion,  and  viewpoint  attitude  remained  fixed  with  respect  to  the  world  such  that 
aircraft  attitude  did  not  dictate  viewpoint  attitude.  Keeping  viewpoint  motion  slaved  to  aircraft 
motion  created  a  rotating  map  for  all  display  condition,  while  fixing  viewpoint  attitude  created 
a  stable  viewing  platform  for  the  display.  The  conditions  are  described  in  greater  detail  here: 

Display  1  was  presented  from  the  perspective  of  the  subject  sitting  in  the  pilots  seat  as 
seen  in  Figure  2.2.  Viewpoint  location  is  considered  fully  egocentric  in  this  display. 
Field  of  view  was  adjusted  such  that  the  amount  of  the  world  in  view  was  the  same  as 
all  other  displays,  FOV  in  this  case  was  set  at  130  degrees  in  the  horizontal,  and  90 
degrees  in  the  vertical.  Viewpoint  attitude  was  fixed  straight  in  front  of  the  aircraft, 
while  azimuth  angle  and  elevation  angle  were  zero  since  the  viewpoint  location  was 
within  the  aircraft. 


39 


Display  2  (Figure  2.3)  was  presented  from  the  perspective  of  the  subject  viewing  the 
aircraft  from  10,000  “display  feet"  from  the  aircraft.  This  exocentric  viewpoint 
location  requires  specification  of  the  viewpoint  location.  In  this  and  both  of  the  other 
exocentric  perspective  displays,  both  the  azimuth  angle  and  the  elevation  angle  were  45 
degrees.  Azimuth  angle  was  adjusted  to  either  45  degrees  or  -45  degrees  (the  right  or 
left  side  of  the  aircraft)  so  that  the  simulated  world  would  always  be  in  the  background. 
The  distance  vector  from  the  viewpoint  to  the  reference  point  (aircraft)  was  10,000 
display  feet,  and  the  FOV  in  this  case  was  120  degrees  and  60  degrees  horizontal  and 
vertical  respectively. 

Display  3  (Figure  2.4)  was  presented  from  the  same  view  as  display  2,  with  the 
exception  that  the  viewing  distance  vector  was  25,000  display  feet.  This  exocentric 
viewpoint  location  required  the  manipulation  of  GFOV  in  order  to  maintain  a 
homogenous  amount  of  the  world  in  the  display;  the  FOV  was  maintained  at  80  degrees 
in  the  horizontal,  and  50  degrees  in  the  vertical. 

Display  4  (Figure  2.5)  was  the  same  view  as  2  and  3  with  the  exception  that  the 
viewing  distance  was  70,000  ft.  FOV  was  further  narrowed  to  45  degrees  in  the 
horizontal,  and  30  degrees  in  the  vertical  in  order  to  maintain  the  quantity  of  the  world 
visible  to  subjects. 

Display  5  (Figure  2.6)  was  a  2  dimensional  plan  view  depiction  of  the  world,  subjects 
were  presented  two  displays,  one  planar  view  of  the  world  to  give  lateral  information, 
and  one  profile  depiction  to  relay  height  information.  Both  views  had  a  fixed  aircraft 
and  a  moving  world. 

Augmentations:  Several  artificial  augmentations  were  incorporated  into  each  display  to 
aid  the  subjects  in  their  flight,  and  can  be  seen  in  Figures  2.2-2.6. 

Each  cardinal  direction  was  clearly  depicted  on  the  world  terrain  with  a  white  arrow, 
and  labeled  appropriately  N,S,E,  or  W. 


In  addition  to  the  actual  flight  path,  a  “margin  of  error”  was  also  included,  which 
represented  100  ft.  of  vertical  or  lateral  deviation  from  the  desired  flight  path.  This  path  was 


presented  in  blue,  and  created  a  perceptual  flight  “box'*  to  judge  error  from  the  desired  flight 
“path”. 

The  aircraft  was  color  coded  to  depict  vertical  deviation  from  the  flight  path.  Five 
color  levels  were  used,  white  to  indicate  >  70  ft  above,  gray  to  indicate  between  70  and  35  ft 
above,  black  to  indicate  a  deviation  of  less  than  35  ft,  yellow  for  between  35  and  70  ft  below, 
and  red  to  indicate  >  70  ft  below  the  flight  path. 

A  directional  predictor  was  added  to  the  display,  which  served  as  an  extension  to  the 
aircraft  and  created  an  extra  long  nose  so  that  subjects  could  better  see  where  they  were 
heading,  both  in  the  vertical  and  the  lateral  axes.  The  end  of  the  nose  and  the  rear  of  the 
aircraft  were  placed  on  “poles”  which  connected  the  aircraft  to  its  shadow  on  the  ground, 
thereby  giving  more  height  information.  This  type  of  augmentation  was  advocated  by  Ellis  et 
al.  (1987). 

Finally,  an  attitude  display  indicator  (ADI)  was  added  at  the  top  center  of  each  display 
to  assist  flight  control.  The  ADI  was  configured  such  that  the  aircraft  symbol  banked  within 
the  display,  while  the  pitch  indicator  bars  moved  to  depict  pitch  inputs.  The  aircraft  symbol  in 
the  ADI  was  white,  while  the  pitch  indicator  bam  were  black. 

Performance  Measurement:  Situation  Awareness  was  assessed  as  guided  by  the  operational 
definition  arrived  at  in  the  introduction.  Both  long  term  and  short  term  memory  of  the 
situation  and  world  were  assessed  using  multiple  choice  questions  referring  to  geographic 
features  and  flight  path  orientation. 

Short  term  memory:  At  a  given  point  on  each  flight  path,  the  simulation  stopped,  the 
screen  was  blacked  out,  and  subjects  were  prompted  a  series  of  questions  to  assess  their 
awareness  of  the  situation  at  that  moment.  Accuracy  and  response  times  were  recorded 
automatically  upon  depressing  a  1,2,3  or  4  on  a  computer  keyboard,  and  subjects  were 
informed  that  both  response  time  and  accuracy  were  important  as  performance  measures. 
Locations  along  each  flight  path  where  the  frozen  screen  tests  occurred  are  depicted  on  Figure 
2.1,  while  the  actual  questions  asked  are  located  in  Table  2.1.  All  eight  questions  were  asked 
at  each  frozen  screen  location.  The  questions  asked  of  subjects  were  derived  from  a  survey 
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administered  to  eight  instructor  qualified  pilots  from  the  University  of  Dlinois,  the  actual 
questionnaire  is  contained  in  Table  2.2.  Overall,  instructors  indicated  that  they  assessed  SA  in 
several  different  ways,  including  but  not  limited  to:  knowledge  of  the  runway  location, 
knowledge  of  future  flight  requirements,  and  knowledge  of  general  location  in  relation  to 
world  objects  and  to  a  desired  location.  The  questions  in  Table  m-1  are  designed  along  these 
specifications.  Note  that  half  of  the  questions  are  specifically  designed  to  assess  the  subjects’ 
knowledge  in  an  ego-referenced  frame  (the  odd  questions)  while  half  of  the  questions  are 
designed  to  assess  knowledge  in  a  world-referenced  frame  (the  even  questions). 

Long  term  memory:  After  completing  the  two  practice  flights  and  the  four  trial  flights, 
subjects  were  given  a  post-test  to  ascertain  how  much  of  the  world  they  had  committed  to  a 
longer  term  memory  store,  as  SA  depends  on  both  information  available  temporarily,  and 
information  pulled  out  of  a  longer  lasting  store  (Figure  2.7).  Subjects  were  given  a  sheet  of 
paper  which  contained  only  the  runway  and  the  cardinal  directions.  They  were  then  given 
paper  cutouts  of  the  objects  in  the  world  and  were  asked  to  place  them  on  the  paper  as  they 
had  appeared  in  the  world.  Subjects  were  evaluated  on  the  correctness  of  their  positioning  of 
the  terrain  features. 

Tracking:  performance  was  recorded  using  RMS  error  and  MAE  from  the  flight  path 
in  terms  of  lateral  deviation  and  vertical  deviation.  Error  was  sampled  at  two  cycles  per 
second  and  averaged  over  3  portions  of  the  flight  path  corresponding  roughly  to  the 
downwind,  base,  and  final  components  of  the  flight  path. 

Procedure 

Upon  arrival,  subjects  were  given  a  demographic  questionnaire  in  order  to  record  age 
and  flight  experience.  Subjects  were  then  given  instructions  on  what  tasks  would  be  required 
of  them  during  the  experiment  (Appendix  A).  All  participants  were  informed  that  their 
participation  was  entirely  voluntary,  and  though  their  participation  was  greatly  valued  they 
could  decline  to  complete  the  experiment  without  forfeiture  of  pay  earned  up  to  that  time.  No 
subjects  declined  to  complete  the  experiment.  After  reading  the  instructions  and  being 
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informed  of  the  nature  of  the  experiment,  subjects  were  seated  in  a  dimly  lit  and  isolated 
room  where  the  apparatus  described  above  was  located. 

Subjects  completed  two  practice  trials  under  close  supervision  of  the  administrator,  and 
were  provided  opportunity  to  ask  any  questions  about  their  particular  display.  The 
administrator  made  sure  that  all  subjects  understood  the  symbology  and  the  display  by  the 
conclusion  of  practice  rounds.  A  door  was  then  closed  and  subjects  were  provided  with  an 
undisturbed  and  isolated  environment  in  which  to  perform  the  experiment.  Recorded  trials 
were  initiated  by  subjects  in  order  to  provide  rest  periods  between  each  trial. 

After  completion  of  all  trials,  subjects  were  given  a  post-test  designed  to  test  long  term 
memory  of  the  terrain,  as  depicted  in  Figure  2.7.  A  subjective  evaluation  was  then 
administered  to  subjects  in  order  to  compile  relative  perceived  difficulty  of  each  of  the  display 
types.  As  Table  2.3  shows,  relative  difficulty  was  rated  on  a  scale  of  1-10,  1.0  being  very 
hard,  while  actual  questions  were  aimed  at  different  aspects  of  the  flight  as  experienced  using 
the  particular  display  type  assigned. 


Figure  2.1:  Layout  of  the  simulated  world  through  which  subjects  had  to  fly.  Line  indicate  flight  paths,  boxes 
iadicate  locations  of  frozen  screen  tasks. 
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£:  i’cr-ipectivc  liispiay  with  an  oxoccntric  view-point  location  70.000  ft.  from  the  aircraft. 

*  ^ 


49 


1.  [ERF]  Where  la  the  runway  to  you? 

1.  right,  front 

2.  left,  front 

3.  right,  behind  (or  even) 

4.  left,  behind  (or  even) 

2.  [WRF]  In  which  general  direction  is  the  aircraft  traveling? 

1.  North 

2.  South 

3.  East 

4.  West 

3.  [ERF]  What  was  the  next  turn  on  the  flight  path  like? 

1.  right,  >  «*  45  degrees 

2.  right,  <  45  degrees 

3.  left,  >  «  45  degrees 

4.  left,  <  45  degrees 

4.  [WRF]  Generally,  where  is  the  runway  from  the  aircraft? 

1.  North 

2.  South 

3.  East 

4.  West 

5.  [ERF]  What  is  your  Aircraft’s  position  relative  to  the  flight  path? 

1 .  above  and  to  the  right  of  the  flight  path 

2.  above  and  to  the  left  of  the  flight  path 

3.  below  and  to  the  right 

4.  below  and  to  the  left 

6.  [WRF]  In  which  direction  will  the  next  turn  take  you? 

1.  more  North 

2.  more  South 

3.  more  East 

4.  more  West 

7.  [ERF]  Is  the  peak  of  the  terrain  feature  located  to  your  [right,  left]  higher  or  lower  than  the  flight  path? 

1.  peak  is  higher 

2.  peak  is  lower 

3.  same  height 

4.  don't  know 

8.  [WRF]  Is  the  peak  of  the  terrain  feature  located  to  the  [N.S.E,  or  W]  higher  or  lower  than  the  flight  path? 

1.  peak  is  higher 

2.  peak  to  lower 

3.  same  height 

4.  don’t  know 


Table  2.1;  Short  Term  Memory  SA  Questions  (during  flight)  odd  questions  are  from  an  Ego-reference  frame, 
while  even  questions  are  from  a  World-reference  frame.  ERF  and  WRF  notations  were  not  present  for  actual 
experimental  presentation. 
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Dear  Instructor, 


A  current  project  being  conducted  at  Aviation  Research  Laboratory  involves  the  study  of 
electronic  approach  displays  as  a  means  to  replace  the  use  of  conventional  paper  plates  such  as 
Jeppesen  charts.  One  important  aspect  being  examined  in  this  study  is  the  concept  of  situation 
awareness.  In  other  words,  how  well  do  the  displays  give  local(the  aircraft  is  above  and  to  my 
right)  and  global(the  thunderstorm  is  west  not  left  of  the  airport)  information.  The  item  to 
examine  is  easy  to  determine,  what  is  not  so  easy  is  how  to  measure  a  concept  such  as 
situation  awareness.  This  is  where  your  experience  is  highly  needed,  please  take  a  few 
moments  to  answer  the  following  question. 

What  are  some  typical  questions  that  you  would  ask  your  students  In  order  to  determine 

*  % 

their  level  of  situation  awareness?  This  includes  In  the  aircraft  and  on  the  ground. 


Your  input  is  greatly  appreciated.  Thank  you. 


Table  2.2;  Instructor  Questionnaire  administered  to  eight  University  of  Illinois  Institute  of  Aviation  instructor 
pilots. 
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SUBJECT# _ 

Using  the  following  scale,  please  answer  the  following  questions. 

Very  easy  Mod.  Easy  not  very  hard  kfrdahard  Very  Hard 
123456789  10 

1.  Controlling  the  aircraft’s  attitude  _ 

2.  Getting  information  off  of  the  terrain  to  answer  questions  _ 

4.  Recalling  if  terrain  features  were  “to  the  right,  left"  _ 

5.  Answering  questions  such  as  “to  the  North  or  South"  _ 

6.  Remembering  what  the  terrain  looked  like  _ 

7.  Estimating  proper  control  inputs  to  stay  on  the  flight  path  _ 

8.  Height  judgments  _ 

9.  Lateral  judgments  _ 

10.  Overall  difficulty  of  the  flying  experience  with  your  type  of  map.  _ 


Table  2.3:  Subjective  Evaluation  forms  used  to  evaluate  perceived  difficulty  of  different  aspects  of  flight  given  one 
of  the  display  conditions. 
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3.  RESULTS 


Tracking  Performance 

Examination  of  deviation  data  in  terms  of  RMS  and  MAE  showed  that  MAE  data  had 
fewer  outliers  and  was  less  skewed,  therefore,  MAE  was  used  to  evaluate  tracking  performance. 
Horizontal  and  vertical  tracking  were  effected  only  by  the  display  type.  There  were  no  trial  or 
path  effects,  and  nno  interaction  between  any  of  the  independent  variables. 

Horizontal:  Deviation  error  in  terms  of  horizontal  MAE  is  presented  in  Figure  3.1.  A 
repeated  measures  ANOVA  with  subjects  nested  in  display  type  was  carried  out,  and  revealed  a 
significant  main  effect  for  map  type  F(4,120)  **106  p<.0001.  In  addition,  a  Tukey  analysis  for 
all  pairwise  comparisons  (TMC  120df,  P<.1  for  family  of  comparisons,  P<.01  for  each 
comparison)  revealed  that  the  egocentric  view  supported  better  performance  than  the  rest  of  the 
maps,  while  the  far-exocentric  view  supported  poorer  performance  than  any  of  the  other  map 
types.  There  was  no  significant  difference  between  the  close  exocentric,  the  mid  exocentric,  or 
the  planar  condition. 

Vertical:  A  similar  analysis  was  performed  on  the  vertical  error  data,  plotted  in  Figure 
3.2.  There  was  a  significant  main  effect  for  map  type  F(4,120)  =  62.7  p<.0001 .  Tukey  multiple 
pairwise  comparisons  analysis  demonstrated  that  the  egocentric  view  and  the  planar  view 
supported  better  vertical  performance  than  the  other  display  types,  while  the  close  exocentric 
view  provided  the  worst  performance  of  all  display  types  (120  df  P<.1  for  the  family  of 
comparisons  P<.01  for  each  comparison).  There  was  no  significant  difference  for  vertical 
tracking  between  far  and  the  mid  exocentric  view,  nor  between  the  planar  and  egocentric  views. 

3D  Exocentric  Isolation:  Figure  3.3  represents  an  isolated  depiction  of  the  influence  of 
perspective  displays  with  exocentric  viewpoint  locations  on  both  horizontal  and  vertical  error. 
This  figure  clearly  demonstrates  a  dimension*  location  interaction  between  the  different  views. 
There  was  a  main  display  effect  for  vertical  performance  (F(2,72)  *  33.1  pc.OOOl),  and  a  main 
display  effect  for  horizontal  performance  (F(2,72)  *  43.17  pc.OOOl).  The  close  exocentric  view 
led  to  worse  vertical  tracking  than  the  mid  or  far  views,  (TMC  72  df,  p<.01  per  comp),  while  the 
far  exocentric  view  led  to  worse  horizontal  tracking  than  either  the  near  or  mid  views.  (TMC  72 
df,  pc.01  per  comp).  This  interaction  suggests  some  form  of  performance  tradeoff,  whereby 
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there  was  more  accurate  horizontal  control  at  relatively  closer  viewpoints,  and  more  accurate 
vertical  control  at  the  furthest  viewpoint. 

Situation  Awareness  (short  term) 

Average  ERF  scores  showed  a  main  effect  of  path  and  no  significant  effects  of  trial,  or 
display  type,  while  the  average  WRF  score  was  influenced  by  map  type  and  by  path.  Average 
response  time  for  STM  questions  as  influenced  by  map  type  and  trial  with  no  significant 
interactions.  These  results  are  examined  in  further  detail  below. 

Display  Effects:  A  Repeated  measures  analysis  was  performed  on  the  average  number 
of  correct  scores,  with  subjects  nested  within  map  type  as  shown  in  Figure  3.4.  There  was  a 
significant  main  effect  of  display  type,  with  average  number  of  correct  WRF  questions  differing 
significantly  as  a  function  of  the  type  of  display  (F(4,120)  "3.21  p<02).  There  was  no  main 
effect  for  ERF  questions,  and  the  only  difference  between  display  types  can  be  attributed  to  the 
difference  between  the  egocentric  display  (worst)  and  the  mid-exocentric  display  (best)  (TMC, 
120  df  P<.01  per  comp). 

There  was  alsc  a  significant  effect  of  display  type  on  response  times  (F(4,120)  *»  4.5 1 
p<.001),  as  shown  in  Figure  3.5.  The  egocentric  and  the  close-exocentric  map  types  supported 
faster  response  times  than  did  the  planar  display.  (TMC  120  df,  P<.01  per  comp). 

Further,  each  short  term  memory  question  was  examined  individually  to  determine  any 
significant  display  main  effects  for  the  type  of  question  asked.  There  were  display  main  effects 
for  three  of  the  eight  questions. 

Question  1 :  Where  is  the  runway  in  relation  to  vour  aircraft?  (North,  South,  East,  or 
West)  The  main  display  effect  for  this  question  was  significant  for  both  accuracy 
(F(4,120)  «  2.89  p<.03)  and  for  response  time  (F(4, 120)  ■  2.63  p<04).  As  seen  in 
Figure  3.6,  the  far  exocentric  and  planar  views  supported  the  most  accurate  performance 
for  this  question,  while  the  egocentric  view  supported  the  least  accurate  performance 
(TMC  120  df ,  P<.01  per  comp).  Subjects’  accuracy  while  using  the  mid  exocentric  and 
close  exocentric  displays  did  not  differ.  In  terms  of  response  time,  individuals  using  the 
mid  exocentric,  and  the  planar  displays  took  longer  to  answer  the  question  than  did 
individuals  using  the  close  exocentric  display  (TMC  120  df  P<.01  per  comp).  Though 
there  appears  to  be  a  speed-accuracy  tradeoff  contained  within  this  question. 
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Question  2:  In  which  general  direction  is  the  aircraft  traveling?  (North,  South,  East  or 
West)  Figure  3.7  plots  both  average  accuracy  and  response  time  for  each  display  type, 
both  of  which  showed  significant  main  effects  for  the  type  of  display  used  (F(4,120)  - 
4.26  p<.003  and  F(4,120)  “  3.84  p<.006).  Thcregocentric  view  led  to  least  accurate 
performance  on  this  question.  Average  performance  by  subjects  using  the  other  map 
types  did  not  differ.  (Tukey,  120df,  P<.Q1  for  each  paired  comparison).  In  addition, 
individuals  using  the  planar  display  on  average  took  longer  to  answer  the  question  than 
individuals  using  the  close  exocentric  view  (TMC  120  df,  P<.01  per  comp.)  Individuals 
using  the  mid,  far,  and  egocentric  views  took  about  the  same  amount  of  time  to  answer 
the  question. 

Question  5:  What  is  vour .-aircraft’s  position  relative  to  the  flight  path?  (above  &  right, 
above  &  left,  below  &  right,  or  below  &  left.)  Figure  3.8  summarizes  average  accuracy 
and  response  times  for  each  display  condition.  The  main  effect  for  accuracy  was 
marginally  significant  (F(4,120)  ”2.16  p<08)  while  the  main  effect  for  response  time 
was  slightly  more  significant  (f(4,120)  »  3.21  p<.015).  The  close  exocentric  view  led  to 
more  accurate  responses  than  the  far  exocentric  view,  while  no  other  comparisons  were 
significant.  (TMC  120  df,  P<.01  per  comp).  In  addition,  participants  using  both  the  far 
and  mid  exocentric  views  took  significantly  longer  than  those  using  the  egocentric  view 
(TMC  120  df,  P<.01  per  comp). 

No  other  questions  showed  significant  differences  in  either  accuracy  or  response  time  for 
the  different  map  displays. 

Path  Effects:  Four  different  paths  were  flown  by  each  subject,  and  the  average  scores  by 
path  are  shown  in  Figure  3.9.  Accuracy  by  question  was  examined  per  path  by  using  a  repeated 
measures  analysis  on  average  scores  for  all  eight  questions.  Average  accuracy  score  on  the 
STM  questions  was  affected  by  path  direction  (F(3, 140)  =  6.7  p<.0003),  with  northbound  paths 
yielding  significantly  higher  average  scores  than  southbound  paths  (TMC  140  df,  P<.01  per 
comp).  There  was  no  main  path  effect  for  average  response  times. 

In  addition,  the  two  types  of  questions  (ERF  and  WRF)  were  examined  separately.  ERF 
and  WRF  performance  by  the  path  flown  are  represented  in  Figure  3.10  There  were  path  main 
effects  for  both  ERF  and  WRF  questions  (F(3,140)  -  4.50  p<.005)  and  (F(3,140)  «=  4.04 
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p<  009)  respectively.  However,  the  accuracy  of  responses  seems  to  be  roughly  equivalent  for 
both  question  types. 

Trial  effects  Trial  main  effects  were  significant  only  for  response  times,  such  that  the 
first  trial  took  significantly  longer  than  the  other  three  tnals  to  answer  the  questions  (TMC,  70  df, 
P<005  per  comp.)  This  effect  was  sustained  through  all  eight  questions. 

Situation  Awareness  (long  term) 

The  map  reconstruction  task  was  evaluated  by  measuring  and  averaging  the  linear 
distance  from  each  subject’s  placement  of  tenain  features  to  the  actual  locations  of  the  features 
within  the  world.  A  repeated  measures  analysis  revealed  that  there  were  no  main  effects  of 
display  type  on  object  placement  Neither  overall  placement  error  nor  placement  error  for  each 
individual  terrain  feature  proved  to  be  significantly  different  for  the  five  display  types.  In 
addition,  height  judgment  information  was  gathered  but  revealed  no  main  display  effects  for 
overall  performance  nor  for  performance  with  individual  terrain  features.  Qualitative  viewing  of 
the  different  questionnaires  revealed  that  subjects  very  often  had  no  idea  where  objects  should  be 
located  and  merely  guessed,  placing  each  object  at  some  arbitrary  location  along  one  of  the  flight 
paths.  In  fact,  certain  subjects  refused  to  complete  the  LTM  task  due  to  its  difficulty  and  their 
speculation  that  they  would  merely  guess  at  correct  locations  and  relative  heights.  Thus,  there 
proved  to  be  a  very  large  floor  effect  due  to  the  fact  that  no  subject  got  very  close  to  perfect 
performance,  and  all  subjects  had  placed  at  least  one  of  the  objects  at  a  very  great  distance  from 
where  it  should  have  been  located. 

Subjective  Evaluations 

Subjective  evaluations  were  requested  only  after  subjects  had  completed  all  four 
recorded  trials.  As  Figure  3.11  indicates,  overall  average  evaluations  showed  that  subjects 
evaluated  the  close  exocentric  view  as  more  difficult  to  use  than  any  of  the  other  exocentric 
views.  (TMC,  35df,  P<.01  per  comp). 

Further,  each  question  requiring  subjective  evaluation  of  difficulty  was  examined 
separately  for  significant  comparisons  between  displays  CTMC,  35  df,  P<01  per  comparison). 
These  results  are  displayed  together  in  Figure  3.12. 
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The  main  display  effect  was  significant  at  F(4,35)  >4.13,  p<008.  Responses  for  the 
egocentric  view  indicated  a  higher  difficulty  than  the  responses  for  either  the  far 
exocentric  view  or  the  mid  exocentric  view.  " 

Question  9:  (difficulty  making  lateral  iiidymenbrt  The  main  display  effect  for  this 
question  was  significant  at  F(4,35)  -  5.93,  p<.0009.  Responses  indicated  a  higher  level 
of  difficulty  for  the  close  exocentric  view  than  for  either  the  mid  exocentric,  the  far 
exocentric  or  the  egocentric  views. 

Question  10:  ( overalLdifficulty  uf  the  flying  experience  with  the  given  map  tvpc'l 
Significant  display  main  effects  were  present  (F(4,35)  -  4.7,  p<.004).  The  egocentric  and 
close  exocentric  views  were  rated  as  more  difficult  than  the  planar  or  far  exocentric 
views. 
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Horizontal  MAE 


Fig  ure  3.1:  Horizontal  error  by  type  of  display 


Eigurai.2:  Vertical  error  by  type  of  display 
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Fiyure  1.S:  Overall  STM  performance  by  map  type  with  response  times 


Fifure  36:  Summary  of  performance  on  question  1 
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L2:  Summary  of  performance  on  question  2 


Figure  3.8:  Summary  of  performance  on  question  S 
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STM  SA  Score  by  Path  Taken 
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Figure  3.9:  Average  score  by  the  path  flown 


Figure  3.10:  Summary  of  Performance  on  SA  questions. 
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Subjective  difficulty  (1-10) 


Figure  3.11:  Average  rating  of  difficulty  for  each  map  type 


Figure  3.12:  Summary  of  Subjective  ratings  for  questions  5, 9,10 


64 


4.  DISCUSSION 


Tracking, .Effects 

The  egocentric  display  supported  better  tracking  on  both  the  lateral  and  vertical  axes  than 

o- 

the  rest  of  the  map  types.  The  superiority  of  the  egocentric  viewpoint  location  to  the  planar 
instruments  in  terms  of  tracking  error  is  consistent  with  Haskell  &  Wickens’  (1993)  findings  and 
hence  supports  our  hypotheses.  In  general,  the  2D  planar  display  supported  better  tracking 
performance  than  the  3D  exocentric  views,  consistent  with  findings  by  Andre  et  al.  (1991),  as 
well  as  Rate  &  Wickens  (1993)  and  Wickens  et  al.  (1994).  The  tracking  advantage  for  2D 
displays  over  the  exocentric  perspective  displays  is  presumably  due  to  ambiguity  of  the 
confounded  lateral  and  vertical  dimensions  in  the  latter  displays,  However,  within  these 
exocentric  displays  tbe  mid-exocentric  viewpoint  distance  seemed  to  be  the  best  compromise 
between  distance  and  distortion,  providing  the  “best  of  both  worlds”,  and  actually  supporting 
tracking  performance  that  was  as  accurate  on  both  axes  as  that  provided  by  the  2D  displays. .  It  is 
also  important  to  note  that  these  results  were  obtained  even  with  cross-coupling  of  pitch  and  roll, 
such  that  the  tracking  task  was  truly  integrated. 

Most  of  the  tracking  performance  differences  between  these  displays  were  expected  and 
understandable.  However,  a  very  interesting  tradeoff  within  the  3D  exocentric  maps  is  apparent 
in  Figure  3.3.  This  tradeoff  indicates  that  the  middle  exocentric  view  supported  better  overall 
tracking  performance  than  either  the  close  view  or  the  far  view  when  both  lateral  and  vertical 
tracking  were  considered  —  a  highly  unexpected  outcome.  One  would  expect  that  as  the 
viewpoint  gets  further  (10-25-70K  feet),  tracking  performance  would  gradually  decrease  due  to 
decreased  resolution  of  the  display  space  in  terms  of  graphic  pixels,  an  effect  documented  by 
Kim  et  al.  (1987).  However,  in  the  present  study  this  decrease  in  resolution  was  partially  offset 
by  the  fact  that  the  GFOV  was  adjusted  in  order  to  keep  the  same  amount  of  terrain  visible  for 
each  of  the  exocentric  perspective  views.  This  effect  can  be  quantified  by  combining  the  GFOV 
and  the  distance  into  one  “compression  factor”.  The  ratio  of  the  viewing  distance  can  be 
specified  as  10-25-70,  and  with  direct  viewing  this  would  mean  that  the  displayed  visual  angle 
subtending  a  given  amount  of  true  lateral  error  (in  feet)  would  decrease  by  the  Arc  tan  function. 
For  an  arbitrary  10  unit  true  distance,  the  Arctan  for  10/10,10/25,  and  10/70  reveals  angles  of  45, 
21,  and  8.1  degrees  respectively.  When  combined  with  the  adjustments  to  the  GFOVs  (120, 80, 
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and  45  deg.),  display  resolution  can  be  identified  by  the  ratio  of  the  Arctan  function  to  the  GFOV 
(45/120  ■  .375;  21/80  ".262;  and  8.1/45  ".18),  with  higher  numbers  indicating  higher 
resolution.  Presumably,  better  performance  can  be  achieved  when  error  resolution  is  high. 

From  these  calculations,  it  is  easy  to  understand  the  high  lateral  error  at  the  for  exocentric 
position,  although  why  error  is  not  increased  at  the  medium  distance  relative  to  the  near  distance 
remains  unclear.  Correspondingly,  it  can  be  understood  that,  since  vertical  error  is  coded  by 
color,  as  well  as  by  spatial  distances,  it  may  very  well  be  immune  to  the  resolution  factor.  This 
color  constancy  across  displays  can  account  for  the  absence  of  vertical  error  differences  between 
the  mid  and  far  exocentric  positions,  but  does  not  explain  the  higher  level  of  vertical  error  at  the 
near  position.  Yeh  &  Silverstein  (1992)  found  a  somewhat  similar  tradeoff  of  tracking  error  type 
by  viewing  distance.  They  found  that  depth  judgments  were  more  accurate  for  a  near  distance, 
and  that  altitude  judgments  were  more  accurate  for  a  far  distance.  We  can  only  conclude  that 
two  different  aspects  of  the  display  configuration  are  impeding  tracking  performance; 

1 .  The  first  is  resolution.  At  far  distances  the  resolution  of  tracking  de  viations  can  be 
relatively  small.  Quite  simply,  it  is  harder  to  see  a  tracking  error  due  to  decreased 
resolution  and  therefore  it  is  harder  to  correct  that  error. 

2.  Perhaps  the  extreme  distortion  caused  by  compressing  a  120  degree  GFOV  into  a 
viewin  g  angle  only  around  60  degrees,  is  sufficiently  disorienting  to  disrupt  accurate 
vertical  tracking  at  close  distances,  negating  any  advantages  that  may  be  present  due  to 
higher  resolution. 

Given  the  effects  of  these  two  aspects  of  display  configuration,  it  can  be  proposed  that  the  middle 
exocentric  viewing  condition  gained  the  advantage  of  higher  resolution,  but  was  sufficiently 
close  to  the  actual  center  of  projection  to  avoid  a  performance  decrease  due  to  distortion. 

Another  sign  of  evidence  for  this  unexpected  tradeoff  effect  can  be  seen  in  subjects’ 
evaluations  of  the  difficulty  of  using  the  maps.  Participants  indicated  that  the  close  exocentric 
view  was  the  most  difficult  to  use,  indicating  that  this  view  imposed  a  high  level  of  workload  and 
attention  to  the  guidance  task.  Endsley  (1993b)  indicates  that  in  some  instances  SA  and 
workload  may  directly  effect  each  other,  and  that  the  key  is  to  identify  situations  which  would 
lead  to  undesirable  combinations  such  as  high  workload  and  low  SA.  In  the  present  study  it 
would  seem  that  extremely  close  exocentric  viewpoints  with  large  fields  of  view  create  higher 
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workload  in  display  interpretation,  while  also  creating  a  lower  level  of  SA;  clearly  situations 
which  should  be  avoided. 

Situation  Awareness 

S  A  effects  were  also  generally  consistent  with  t£e  hypotheses  proposed  in  the 
introduction.  .  The  highest  average  accuracy  scores  of  short  term  SA  were  attained  with 
exocentric  viewpoint  locations,  particularly  at  the  middle  distance.  The  critical  consideration 
here  is  that  3D  perspective  displays  provided  better  performance  than  2D  planar  displays  with  the 
same  information.  This  outcome  seems  appropriate  due  to  the  ecological  nature  of  3D  displays. 
Several  studies  (Ellis  et  al.,  1987;  Bemis  et  al.,  1988;  Wise  et  al.,  1993)  found  similar 
advantages  for  exocentric  perspective  3D  displays.  By  the  same  token,  it  is  important  to  note 
that  the  awareness  advantage  for  3D  displays  was  limited  to  the  exocentric  viewpoint  locations, 
and  that  awareness  at  the  egocentric  viewpoint  location  was  poorer  than  awareness  for  subjects 
using  the  planar  2D  displays. 

Also,  results  found  by  Barfield  et  al.  (1992)  were  replicated  in  that  all  of  the  god’s  eye 
perspectives  outperformed  the  fully  egocentric  display  in  terms  of  response  accuracy  for  short 
term  situation  awareness.  Although,  there  was  a  slight  response  time  advantage  for  the 
egocentric  and  near  exocentric  viewpoint  locations  as  shown  in  Figure  3.5.  Surprisingly,  we 
found  no  selective  advantage  for  the  egocentric  display  on  ERF  questions.  We  might  have 
predicted  that  due  to  the  pilots  eye  view,  ego  referenced  questions  would  be  entirely  consistent 
with  the  participants  view,  and  this  advantage  would  have  been  reflected  in  more  accurate 
responses  to  the  ERF  questions.  However,  it  is  likely  that  we  found  no  egocentric  viewpoint 
advantage  for  ERF  questions  due  to  the  fact  that  exocentric  viewpoints  were  only  exocentric  in 
terms  of  viewpoint  location,  and  that  the  viewpoint  motion  and  orientation  were  linked  to  the 
aircraft  motion  such  that  even  these  displays  were  somewhat  egocentric.  Thus,  rotating  maps 
were  created  for  all  of  the  displays  that  allowed  ERF  questions  to  be  answered  without  mental 
alignment  of  the  display  to  the  ERF. 

The  individual  STM  SA  questions  revealed  some  interesting  trends  as  well.  In  question 
1,  Figure  3.6  demonstrates  the  classic  speed  accuracy  tradeoff  seen  in  so  many  different 
situations.  For  this  reason,  knowing  where  the  runway  is  in  relation  to  the  aircraft  seems  not  to 
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be  effected  much  by  map  type,  even  thoughthere  are  significant  differences  in  the  average  score 
of  participants  using  different  display  types. 

In  contrast  to  the  tradeoff  seen  in  question  1,  examination  of  questions  2  and  5  reveals  a 
more  clear  interpretation.  Responses  to  question  2  asking  the  momentary  direction  of  aircraft 
flight  (Figure  3.7)  showed  very  little  interaction  of  speed  and  accuracy.  Subjects  flying  the 
egocentric  view  simply  did  not  know  which  direction  they  were  traveling,  as  their  accuracy  was 
around  chance.  A  trade-off  does  appear  to  exist  within  the  exocentric  3D  displays,  the  mid 
exocentric  viewpoint  showing  lower  error  rates,  but  higher  response  times.  In  addition, 
contrasting  the  far  exocentric  with  the  planar  display  did  not  reveal  a  change  in  error  rate,  but  did 
show  the  highest  response  times  with  the  planar  display.  In  agreement  with  the  overall  SA 
performance,  it  would  appear  that  performance  on  question  2  taking  into  account  both  speed  and 
accuracy  was  best  using  the  3D  exocentric  displays.  Question  5,  assessing  subjects'  knowledge 
of  their  lateral  and  vertical  location  relative  to  the  flight  path,  shows  the  greatest  disparity 
between  displays  for  the  individual  questions.  There  is  almost  no  evidence  of  a  speed  accuracy 
tradeoff  in  this  question,  the  egocentric  and  close  exocentric  view  preserved  subjects'  knowledge 
better  than  did  the  mid  exocentric,  the  far  exocentric,  and  the  planar  displays. 

One  somewhat  surprising  finding  was  the  absence  of  any  effect  of  map  dimensionality  on 
questions  7  and  8.  These  two  questions  required  that  subjects  judge  the  height  of  a  near  terrain 
feature  relative  to  the  aircraft.  On  the  one  hand,  an  advantage  for  the  3D  display  could  have 
resulted  due  to  increased  integration  of  the  visual  scene.  On  the  other  hand,  we  might  have 
predicted  that  the  2D  displays  would  have  supported  more  accurate  performance  than  the  3D 
displays  due  to  the  fact  that  this  judgment  required  focused  attention  to  only  the  vertical  axis.  As 
predicted  by  Haskell  &  Wickens  (1993)  this  focused  attention  task  should  be  better  supported  by 
2D  displays  which  do  not  have  the  drawback  of  axis  confusion.  We  can  assume  here  that  the 
advantage  of  the  integrated  3D  rendering  may  have  been  offset  by  this  greater  precision  and 
reduced  ambiguity  while  using  the  profile  portion  of  the  perspective  display,  and  the  end  result 
was  a  canceling  of  the  different  advantages  of  each  display  type. 

Path  effects  were  found  when  Northbound  paths  were  compared  with  Southbound  paths. 
The  reduced  error  when  traveling  north  is  consistent  with  the  findings  of Wickens  et  ai.  (1994) 
and  many  other  studies  (e.g.  Aretz,  1991;  Harwood  &  Wickens,  1991).  The  northbound 
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advantage  can  be  attributed  to  the  need  to  mentally  rotate  the  southbound  paths  in  order  to 
answer  geographical  questions.  However,  the  northbound  advantage  was  curiously  evident  in 
both  WRF  and  ERF  questions.  The  accuracy  cost  suffered  by  Southbound  paths  for  WRF 
questions  is  very  understandable  and  predictable,  as  the  forward  view  is  in  a  Southerly 
orientation,  and  responses  require  mental  rotation  to  align  the  instantaneous  heading  with  true 
North.  However,  ERF  questions  are  dependent  on  only  the  instantaneous  heading  of  the  aircraft, 
and  do  not  have  to  be  translated  to  a  World-reference  frame.  The  reason  an  advantage  for 
Northbound  paths  in  terms  of  ERF  questions  remains  puzzling.  Two  possible  solutions  may  be 
offered: 

1 .  It  is  possible  that  traveling  North  does  not  requirre  constant  resource  dedication  to 
maintaining  awareness  of  the  WRF,  as  subjects  can  directly  perceive  information.  This 
leads  to  more  processing  resources  available  to  keep  alert  of  ERF  information. 

2.  It  could  be  that  the  questions  were  simply  posed  from  locations  along  the  Northbound 
paths  which  were  easier  to  interpret.  Examination  of  Figure  2.1  reveals  that  frozen  screen 
locations  for  Northbound  paths  took  place  when  headings  were  directly  aligned  with 
world  coordinates  (either  North  or  West),  while  locations  for  the  frozen  screen  task  on 
southbound  paths  were  at  headings  not  consistent  with  world  cardinal  directions.  It  is 
recommended  that  further  studies  be  accomplished  with  multiple  locations  for  SA  tasks 
to  avoid  possible  confounds  of  this  nature. 

In  addition  to  SA  effects  due  to  the  map  flown  and  the  path  taken,  there  was  one 
significant  main  effect  of  trial;  response  times  in  the  first  trial  were  substantially  longer  than  any 
other  trials.  This  effect  can  be  directly  related  to  the  fact  that  participants  did  not  have  any 
practice  on  S  A  questions,  and  the  first  trial  was  the  first  time  they  had  been  exposed  to  the 
questions.  Since  these  questions  were  identical  in  each  path,  the  fact  that  response  time 
decreased  with  practice  is  hardly  surprising.  Subjects  had  two  practice  trials  for  tracking,  which 
would  explain  why  there  was  no  parallel  trial  effect  for  tracking  performance. 

While  the  results  for  STM  S  A  measures  revealed  many  significant  and  meaningful 
effects,  the  lack  of  significant  results  for  LTM  SA  was  disappointing.  Overall  map  recall  scores 
were  very  poor,  and  there  were  no  LTM  differences  due  to  display  type  at  all,  indicating  no 
difference  between  the  long  term  memory  store  of  the  terrain.  The  possibility  exists  that  subjects 
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simply  did  not  have  the  duration  of  exposure  to  the  experimental  world  and  were  focused  too 
much  on  local  guidance  to  encode  many  of  the  objects  located  within  the  domain;  an  effect 

which  may  also  be  apparent  in  actual  flight  missions.  Also,  all  maps  were  rotating  and  hence 

«*• 

would  be  predicted  on  the  basis  of  Aretz’s  (1991)  findings  to  fair  poorly  on  this  test  because  of 
the  inconsistency  of  the  location  of  landmark  depiction  across  flights.  The  LTM  recall  task  was 
presented  specifically  in  terms  of  world  referenced  coordinates,  and  subjects  had  only  observed 
landmarks  through  an  ERF.  Alternative  methods  of  gaining  information  as  to  LTM  store  could 
involve  decreasing  the  number  of  terrain  features,  or  increasing  exposure  to  the  experimental 
domain.  Either  way,  there  i  s  a  need  to  evaluate  the  LTM  S  A  imparted  to  pilots,  and  the  present 
study  failed  to  achieve  a  good  measure  of  this  construct 
Subjective.  Evaluations 

The  results  of  subjective  evaluations  indicate  that  participants  using  the  egocentric  and 
close  exocentric  displays  felt  those  displays  were  difficult  and  demanding  to  use.  This  influence 
on  perceived  difficulty  did  not  exactly  match  tracking  performance,  as  tracking  was  actually  best 
with  the  egocentric  viewpoint  location.  Subjective  ratings  of  difficulty  were  also  in  partial 
disagreement  with  STM  SA  response  latency.  While  subjects  indicated  that  the  fully  egocentric 
view  and  the  close  exocentric  view  were  the  most  difficult  to  use,  they  responded  more  rapidly 
than  did  subjects  using  the  other  display  types. 

While  this  could  be  due  to  subjects  simply  guessing,  we  believe  this  is  not  the  case,  as 
subjects  performance  on  the  short  term  memory  questions  was  above  chance.  Endsley  (1993b) 
characterizes  SA  and  workload  as  flip  sides  of  the  same  coin,  asserting  that  “Workload  is 
generated  directly  from  the  effort  it  takes  to  achieve  and  maintain  SA”.  While  this  statement 
appears  to  be  somewhat  true  in  this  case,  as  the  displays  with  the  highest  subjective  workload 
had  the  quickest  response  times,  it  is  not  totally  true,  as  subjects  using  the  middle  distance 
exocentric  viewpoint  the  greatest  accuracy,  but  did  not  think  the  task  was  the  most  difficult. 
More  research  including  the  gathering  of  both  subjective  workload  and  awareness  should  be 
accomplished  to  further  quantify  the  relationship  between  the  two. 

Overall  Conclusions 

First  and  foremost  of  our  conclusions  is  that  properly  designed  3D  displays  can 
outperform  2D  displays  for  navigation  and  for  S A.  However,  this  conclusion  is  dependent  on  the 


70 


type  of  3D  display  used;  fully  egocentric  locations  support  better  tracking  performance,  and 
exocentric  locations  support  better  SA  performance.  Also,  not  all  3D  exoeentric  displays  support 
higher  SA.  Only  displays  such  as  the  mid  exocentric  distance  are  advantageous  because  they 
ovoid  low  resolution  at  far  distances,  and  FOV  distortions  at  close  distances.  Further 
examination  of  these  3D  exocentric  views  in  actual  flying  tasks  is  essential  in  order  to  define  the 
exact  location  form  which  to  display  the  map. 

Within  the  four  3D  display  prototypes,  Figure  4.1  depicts  what  appears  to  be  a  tradeoff 
between  displays  that  support  local  guidance  (the  average  distance  from  the  flight  path)  and 
displays  that  support  global  situation  awareness  (the  error  rate  on  STM  SA  questions)  when  the 
viewpoint  is  moved  from  the  most  ego-referenced  to  a  more  exocentric  reference  frame.  In 
Figure  4.1  the  left-hand  ordinate  is  tracking  error  as  derived  from  average  horizontal  and  vertical 
error,  the  right  ordinate  is  scaled  for  average  accuracy  on  short  term  awareness  questions. 
Whether  the  fully  egocentric  display  or  the  mid-exocentric  display  is  “better”  in  this  comparison 
depends  upon  whether  one  gives  greater  weighting  to  local  guidance  (egocentric)  or  global 
awareness  (mid-exo).  Given  this  tradeoff  between  global  SA  and  Local  Guidance  revealed  as  the 
degree  of  egocentrism  in  the  display  increases,  it  would  seem  that  the  ideal  use  of  electronic 
aviation  displays  for  guidance  to  landing  should  incorporate  a  3D  exocentric  view  for  higher 
levels  of  the  flight  path  where  exact  guidance  is  not  quite  so  important,  but  that  for  approaches 
close  to  the  ground,  where  guidance  is  extremely  important,  a  3D  egocentric  display  proves  to 
best  support  accurate  guidance. 

An  important  limitation  to  the  generalizability  of  the  results  of  present  study,  is  that 
performance  was  assessed  while  only  using  an  electronic  display,  and  not  in  an  actual  flying 
environment,  and  hence  characterizing  instrument  meteorological  conditions.  The  displays  could 
interact  very  differently  if  used  in  conjunction  with  other  flight  instruments,  or  with  an  actual 
forward  field  of  view  as  in  visual  contact  conditions,  where  ERF  to  WRF  transitions  are  required 
to  maintain  controlled  flight  Nevertheless,  given  discussions  in  further  development  of  zero 
visibility  landing  displays  for  pilots,  the  current  evaluation  has  a  great  dial  of  potential  relevance. 
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Overall  Performance 
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Figure  4.1:  Summary  of  performance,  both  in  terms  of  SA  and  LG  Tradeoff  is  apparent  in  the  interaction  of  LG  and 
SA.  Data  is  for  overall  score  on  STM  SA  tasks  and  tracking  error  as  defined  by  average  horizontal  and  vertical 
MAE. 
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Appendix  A 

PLEASE  READ  THESE  INSTRUCTIONS  CAREFULLY 


The  Reason 

The  conventional  paper  approach  plates  are  cluttered,  and  have  been  occasional  sources 
of  geographical  disorientation  and  high  cockpit  workload.  In  addition,  such  paper  plates  are 
difficult  and  expensive  to  update  or  change  in  accordance  with  a  new  flight  path  in  the  terminal 
area.  Electronic  displays  in  the  cockpit  bring  the  potential  to  substantially  redesign  the 
approach  plate  format,  and  our  research,  supported  by  NASA,  is  intestded  to  support  this 
redesign  effort.  The  purpose  of  this  experiment  is  to  evaluate  different  types  of  electronic 
maps,  some  of  which  depart  considerably  from  the  conventional  type  of  paper  approach  plate. 

You  are  to  fly  a  total  of  six  approaches.  The  first  two  will  be  practice  runs,  and  the 
last  four  will  be  recorded.  You  will  fly  each  of  four  different  approaches  to  the  same  runway, 
in  the  same  simulated  airport  area. 

The  Displays 

You  will  fly  one  of  five  different  display  types: 

1)  sitting  in  the  pilots  seat,  looking  out  the  windscreen  (figure  2.2) 

2)  outside  viewing  of  the  aircraft  from  a  near  distance  (figure  2.3) 

3)  outside  viewing  of  the  aircraft  from  a  medium  distance  (figure  2.4) 

4)  outside  viewing  of  the  aircraft  from  a  far  distance  (figure  2.5) 

3)  viewing  the  aircraft  in  a  planar  or  bird’s  eye  view.(figure  2.6) 

The  task,  is  the  same  for  the  different  display  types,  however,  the  visual  representations 
are  what  changes.  Also,  the  same  amount  of  the  simulated  terminal  area  will  be  displayed 
regardless  of  which  type  of  map  you  use. 

Your  Task 

You  will  see  a  flight  path  and  obstacles.  Your  job  is  two-fold.  First,  keep  the  aircraft 
as  close  to  the  flight  path  as  you  can,  and  second,  remain  aware  of  the  geographic  and 
dynamic  situations,  (i.e.  know  where  you  are  on  the  flight  path,  and  where  important  terrain 
features  are  around  you.) 

To  Help  You 

There  are  several  items  added  to  the  display  to  help  you  fly.  l)There  is  “flight  box” 
around  the  flight  path,  which  shows  you  an  alley  down  which  you  are  supposed  to  fly.  2)The 
aircraft  changes  color  if  you  are  too  high  or  too  low:  red  for  very  low,  yellow  for  a  little  low, 
black  if  you  are  acceptably  close  to  the  flight  path,  gray  if  you  are  a  little  too  high,  and  white 
if  you  are  very  high.  3)There  is  a  predictor  present  which  shows  you  where  the  aircraft’s  nose 
is  pointed.  4)Cardinal  directions  (N,S,E,W)  axe  displayed  on  the  terrain  with  appropriate 
white  arrows  showing  the  respective  direction.  5)FinaUy,  there  is  an  ADI  available  for  your 
use,  it  is  set  up  with  a  moving  aircraft  and  a  fixed  horizon. 
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How.  You  Will  be  Assessed 

The  computer  will  record  your  deviations  from  the  flight  path,  both  vertically  and 
laterally.  In  addition,  at  any  time  during  each  flight,  the  simulation  will  stop  and  you  will  be 
prompted  to  answer  eight  multiple  choice  questions  designed  to  test  your  knowledge  of  the 
terrain  and  your  current  situation.  On  these  questions^  respond  appropriately  1,2,3  or  4;  both 
your  response  time,  and  your  accuracy  will  be  recorded.  After  completion  of  all  your 
approaches,  you  will  be  asked  to  reconstruct  the  terrain  that  you  flew  through. 


Due  to  the  fact  that  all  displays  show  the  same  amount  of  the  terrain  at  different 
locations,  some  of  them  are  subject  to  distortions  which  you  might  not  be  familiar  with.  The 
Dynamic  flight  environment  of  the  simulation  DOES  NOT  necessarily  represent  what  you 
would  actually  see  if  you  had  a  windscreen  to  look  out.  The  distortion  is  somewhat  like  the 
effect  of  looking  out  a  front  door  peep  hole,  where  you  can  see  quite  a  bit  of  the  hallway,  but 
it  looks  very  distorted  (although  not  quite  as  drastic).  In  terms  of  the  computer  screen,  if  you 
point  to  objects  on  the  computer  screen,  that  will  not  necessarily  correlate  with  where  you 
would  point  to  the  object  if  you  were  looking  out  of  a  windscreen. 
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